Imaging apparatus

ABSTRACT

An imaging apparatus includes a first imaging element that receives light having passed through a first optical element and outputs an image which has a luminance value corresponding to intensity of the light and is processed as a reference image. A second imaging element receives light having passed through a second optical element and outputs an image which has a luminance value corresponding to intensity of the light and is processed as a reference image. A distance calculation device calculates a distance image on the basis of the reference image and the comparison image, and recognition device recognizes an object on the basis of the distance image calculated by the distance calculation apparatus, in which the optical elements or the imaging elements satisfy at least one of conditions in which transmittance of the first optical element is higher than transmittance of the second optical element.

TECHNICAL FIELD

The present invention relates to an imaging apparatus which calculates adistance image (range image) from a plurality of captured images.

BACKGROUND ART

In the related art, as disclosed in PTL 1, an imaging element for atemplate image (reference image) has sensitivity on a longer wavelengthside than an imaging element for a searched image (comparison image) incalculation of a distance image; three-dimensional information of anobject is calculated by using a function which defines a relationshipbetween focal length information or aberration information and awavelength component; and three-dimensional measurement accuracy isprevented from deteriorating due to variations in a focal length, anaberration characteristic, or the like for each wavelength in a widewavelength band.

CITATION LIST Patent Literature

PTL 1: International Publication No. WO2011/083669

SUMMARY OF INVENTION Technical Problem

Imaging elements have variations in sensitivity characteristics, noise,defective pixels, and the like for each imaging element. In addition,lenses have variations in transmittance, distortion, and the like foreach lens. For this reason, in PTL 1, due to performance variations ofthe imaging elements and the lenses, quality of an image used for arecognition process varies, and recognition performance varies for eachof the imaging apparatuses.

From the above description, an object of the present invention is toprovide an imaging apparatus capable of improving recognitionperformance and reducing variations in the recognition performance foreach imaging apparatus.

Solution to Problem

In order to solve the above-described problem, according to the presentinvention, there is providing an imaging apparatus including a firstoptical element; a first imaging element that receives light havingpassed through the first optical element, and outputs an image which hasa luminance value corresponding to intensity of the light and isprocessed as a reference image;

a second optical element; a second imaging element that receives lighthaving passed through the second optical element, and outputs an imagewhich has a luminance value corresponding to intensity of the light andis processed as a reference image; distance calculation means forcalculating a distance image on the basis of the reference image and thecomparison image; and recognition means for recognizing an object on thebasis of the distance image, in which the first optical element and thesecond optical element, or the first imaging element and the secondimaging element satisfy at least one of conditions in whichtransmittance of the first optical element is higher than transmittanceof the second optical element; a distortion of the first optical elementis smaller than a distortion of the second optical element; asensitivity characteristic of the first imaging element is higher than asensitivity characteristic of the second imaging element; a level ofnoise of the first imaging element is lower than a level of noise of thesecond imaging element; the number of defective pixels of the firstimaging element is smaller than the number of defective pixels of thesecond imaging element; a sensitivity characteristic of the firstimaging element which receives light having passed through the firstoptical element is higher than a sensitivity characteristic of thesecond imaging element which receives light having passed through thesecond optical element; and a level of noise of the first imagingelement which receives light having passed through the first opticalelement is lower than a level of noise of the second imaging elementwhich receives light having passed through the second optical element.

In addition, there is provided an imaging apparatus including a firstoptical element; a first imaging element that receives light havingpassed through the first optical element, and outputs a first imagewhich has a luminance value corresponding to intensity of the light; asecond optical element; a second imaging element that receives lighthaving passed through the second optical element, and outputs a secondimage which has a luminance value corresponding to intensity of thelight; reference image selection means for selecting one imagesatisfying a predetermined condition, of the first image and the secondimage, as a reference image, and selecting the other image as acomparison image; distance calculation means for calculating a distanceimage on the basis of the reference image and the comparison image; andrecognition means for recognizing an object on the basis of the distanceimage calculated by the distance calculation means, in which thepredetermined condition in the reference image selection means isrelated to at least one of an image in which transmittance is higherwhen the transmittance of the first optical element is compared with thetransmittance of the second optical element; an image in which adistortion is smaller when the distortion of the first optical elementis compared with the distortion of the second optical element; an imagein which a sensitivity characteristic is higher when the sensitivitycharacteristic of the first imaging element is compared with thesensitivity characteristic of the second imaging element; an image inwhich a level of noise is lower when the level of noise of the firstimaging element is compared with the level of noise of the secondimaging element; an image in which the number of defective pixels issmaller when the number of defective pixels of the first imaging elementis compared with the number of defective pixels of the second imagingelement; an image in which a sensitivity characteristic is higher whenthe sensitivity characteristic of the first imaging element whichreceives light having passed through the first optical element iscompared with the sensitivity characteristic of the second imagingelement which receives light having passed through the second opticalelement; and an image in which a level of noise is lower when the levelof noise of the first imaging element which receives light having passedthrough the first optical element is compared with the level of noise ofthe second imaging element which receives light having passed throughthe second optical element.

Further, there is provided an imaging apparatus including a firstoptical element; a first imaging element that receives light havingpassed through the first optical element, and outputs a first imagewhich has a luminance value corresponding to intensity of the light; asecond optical element; a second imaging element that receives lighthaving passed through the second optical element, and outputs a secondimage which has a luminance value corresponding to intensity of thelight; characteristic storage means for storing at least one piece ofcharacteristic information such as distortions of the first opticalmeans and the second optical means, sensitivity characteristics, levelsof noise, and the number of defective pixels of the first imaging meansand the second imaging means, and sensitivity characteristics and thelevels of noise of the first imaging means which receives light havingpassed through the first optical means and the second imaging meanswhich receives light having passed through the second optical means;reference image selection means for selecting one image satisfying apredetermined condition, as a reference image, and selecting the otherimage as a comparison image, on the basis of the characteristicinformation stored in the characteristic storage means; distancecalculation means for calculating a distance image on the basis of thereference image and the comparison image; and recognition means forrecognizing an object on the basis of the distance image calculated bythe distance calculation means, in which the predetermined condition inthe reference image selection means is related to at least one of animage in which transmittance is higher when the transmittance of thefirst optical element is compared with the transmittance of the secondoptical element; an image in which a distortion is smaller when thedistortion of the first optical element is compared with the distortionof the second optical element; an image in which a sensitivitycharacteristic is higher when the sensitivity characteristic of thefirst imaging element is compared with the sensitivity characteristic ofthe second imaging element; an image in which a level of noise is lowerwhen the level of noise of the first imaging element is compared withthe level of noise of the second imaging element; an image in which thenumber of defective pixels is smaller when the number of defectivepixels of the first imaging element is compared with the number ofdefective pixels of the second imaging element; an image in which asensitivity characteristic is higher when the sensitivity characteristicof the first imaging element which receives light having passed throughthe first optical element is compared with the sensitivitycharacteristic of the second imaging element which receives light havingpassed through the second optical element; and an image in which a levelof noise is lower when the level of noise of the first imaging elementwhich receives light having passed through the first optical element iscompared with the level of noise of the second imaging element whichreceives light having passed through the second optical element.

Advantageous Effects of Invention

According to the present invention, it is possible to provide an imagingapparatus capable of improving recognition performance and reducingvariations in the recognition performance for each imaging apparatus.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating a configuration of an imaging apparatusaccording to an embodiment of the present invention.

FIG. 2 is a diagram illustrating an operation example of the imagingapparatus of FIG. 1.

FIG. 3 is a diagram illustrating a reference image and a comparisonimage in the imaging apparatus of the present invention.

FIG. 4 is a diagram illustrating a configuration of an imaging apparatusaccording to another embodiment of the present invention.

FIG. 5 is a diagram illustrating an operation example of the imagingapparatus of FIG. 4.

FIG. 6 is a diagram illustrating another operation example of theimaging apparatus of FIG. 4.

DESCRIPTION OF EMBODIMENTS Embodiment 1

FIG. 1 illustrates a configuration of an imaging apparatus according toan embodiment of the present invention.

The imaging apparatus according to the embodiment of the presentinvention includes an imaging unit (first imaging unit) 100 a, animaging unit (second imaging unit). 100 b, a calculation unit 110, ascreen/sound output unit 130, and a control unit 140.

The imaging unit 100 a such as a camera includes an optical element(first optical element) 101 a, shutter means (first shutter means) 102a, an imaging element (first imaging element) 103 a, and characteristicstorage means 104 a.

The optical element 101 a such as a lens refracts light and forms animage on the imaging element.

The shutter means 102 a such as a shutter, which is provided at alocation through which light having passed through the optical element101 a passes, opens a shutter mechanism so that the light passestherethrough only for an exposure time during photographing, and closesthe shutter mechanism so that the light is blocked for a non-exposuretime.

The imaging element 103 a receives an image of the light refracted bythe optical element 101 a, and generates an image corresponding tointensity of the light.

The characteristic storage means 104 a stores information regardingtransmittance, distortion, and the like of the optical element 101 a,information regarding a sensitivity characteristic, noise, the number ofdefective pixels, and the like of the imaging element 103 a, andinformation regarding a sensitivity characteristic, noise, and the likeof the imaging unit 100 a. The information regarding distortion of theoptical element 101 a includes a distortion coefficient of a lens in aradial direction, a distortion coefficient of a lens in a tangentialdirection, and the like. The information regarding a sensitivitycharacteristic of the imaging element 103 a includes a dynamic range, aluminance value of an image obtained by imaging an object with uniformlight, and the like. The information regarding noise of the imagingelement 103 a includes an SN ratio, a standard deviation (variation) ofluminance values of an image obtained by imaging an object with uniformlight, shot noise of light with predetermined intensity, dark currentnoise, reading noise, fixed pattern noise of light with predeterminedintensity, and the like. The information regarding a sensitivitycharacteristic of the imaging unit 100 a includes a dynamic range of theimaging element 103 a which receives light having passed through theoptical element 101 a, a luminance value of an image of the imagingelement 103 a which receives light having passed through the opticalelement 101 a when an object is imaged with uniform light, and the like.The information regarding noise of the imaging unit 100 a includes an SNratio of the imaging element 103 a which receives light having passedthrough the optical element 101 a, a standard deviation (variation) ofluminance values of an image of the imaging element 103 a which receiveslight having passed through the optical element 101 a when an object isimaged with uniform light, shot noise of the imaging element 103 a whichreceives light having passed through the optical element 101 a whenlight with predetermined intensity is incident, dark current noise ofthe imaging element 103 a which receives light having passed through theoptical element 101 a, reading noise of the imaging element 103 a whichreceives light having passed through the optical element 101 a, fixedpattern noise of the imaging element 103 a which receives light havingpassed through the optical element 101 a when light with predeterminedintensity is incident, and the like.

The imaging unit 100 b such as a camera includes an optical element(second optical element) 101 b, shutter means (second shutter means) 102b, an imaging element (second imaging element) 103 b, and characteristicstorage means 104 b. In addition, design values of focal lengths of theimaging unit 100 a and the imaging unit 100 b are the same as eachother. Directions of optical axes of the imaging unit 100 a and theimaging unit 100 b are substantially the same as each other.

The optical element 101 b such as a lens refracts light and forms animage on the imaging element.

The shutter means 102 b such as a shutter, which is provided at alocation through which light having passed through the optical element101 b passes, opens a shutter mechanism so that the light passestherethrough only for an exposure time during photographing, and closesthe shutter mechanism so that the light is blocked for a non-exposuretime.

The imaging element 103 b receives an image of the light refracted bythe optical element 101 b, and generates an image corresponding tointensity of the light.

The characteristic storage means 104 b stores information regardingtransmittance, distortion, and the like of the optical element 101 b,information regarding a sensitivity characteristic, noise, the number ofdefective pixels, and the like of the imaging element 103 b, andinformation regarding a sensitivity characteristic, noise, and the likeof the imaging unit 100 b. The information regarding distortion of theoptical element 101 b includes a distortion coefficient of a lens in aradial direction, a distortion coefficient of a lens in a tangentialdirection, and the like. The information regarding a sensitivitycharacteristic of the imaging element 103 b includes a dynamic range, aluminance value of an image obtained by imaging an object with uniformlight, and the like. The information regarding noise of the imagingelement 103 b includes an SN ratio, a standard deviation (variation) ofluminance values of an image obtained by imaging an object with uniformlight, shot noise of light with predetermined intensity, dark currentnoise, reading noise, fixed pattern noise of light with predeterminedintensity, and the like. The information regarding a sensitivitycharacteristic of the imaging unit 100 b includes a dynamic range of theimaging element 103 b which receives light having passed through theoptical element 101 b, a luminance value of an image of the imagingelement 103 b which receives light having passed through the opticalelement 101 b when an object is imaged with uniform light, and the like.The information regarding noise of the imaging unit 100 b includes an SNratio of the imaging element 103 b which receives light having passedthrough the optical element 101 b, a standard deviation (variation) ofluminance values of an image of the imaging element 103 b which receiveslight having passed through the optical element 101 b when an object isimaged with uniform light, shot noise of the imaging element 103 b whichreceives light having passed through the optical element 101 b whenlight with predetermined intensity is incident, dark current noise ofthe imaging element 103 a which receives light having passed through theoptical element 101 b, reading noise of the imaging element 103 b whichreceives light having passed through the optical element 101 b, fixedpattern noise of the imaging element 103 b which receives light havingpassed through the optical element 101 b when light with predeterminedintensity is incident, and the like.

The imaging unit 100 a and the imaging unit 100 b include the opticalelement 101 a and the optical element 101 b or the imaging element 103 aand the imaging element 103 b, satisfying one item a among the followingitems 1-1 to 1-7 which are set in advance. In a case where a pluralityof imaging apparatuses are manufactured, the imaging apparatuses areconfigured to satisfy an item a which is a predetermined condition.

-   -   Item 1-1: The transmittance of the optical element 101 a is        higher than that of the optical element 101 b.    -   Item 1-2: The distortion of the optical element 101 a is smaller        than that of the optical element 101 b.    -   Item 1-3: The sensitivity characteristic of the imaging element        103 a is higher than that of the imaging element 103 b.    -   Item 1-4: The level of noise of the imaging element 103 a is        lower than that of the imaging element 103 b.    -   Item 1-5: The number of defective pixels of the imaging element        103 a is smaller than that of the imaging element 103 b.    -   Item 1-6: The sensitivity characteristic of the imaging unit 100        a is higher than that of the imaging unit 100 b.    -   Item 1-7: The level of noise of the imaging unit 100 a is lower        than that of the imaging unit 100 b.

The calculation unit 110 constituted by a central processing unit (CPU),a memory, and the like includes reference image storage means 111,comparison image storage means 112, processed image storage means 113,luminance correction information storage means 114, geometricrectification information storage means 115, synchronization signaltransmission means 116, reference image acquisition means 117 a,comparison image acquisition means 117 b, luminance correction means118, geometric rectification means 119, disparity calculation means 120,distance calculation means 121, recognition means 122, andcharacteristic input/output means 123.

The reference image storage means 111 such as a memory or hard diskstores an image captured by the imaging unit 100 a. In disparitycalculation, a template image is cut out from the image stored in thereference image storage means 111, and thus the stored image is areference image.

The comparison image storage means 112 such as a memory or a hard diskstores an image captured by the imaging unit 100 b. In the disparitycalculation, the image stored in the comparison image storage means 112is searched for by using a template image, and thus the stored image isa comparison image.

The processed image storage means 113 such as a memory, or a hard diskstores an image which is processed and generated by the calculation unit110.

The luminance correction information storage means 114 such as a memoryor a hard disk stores a luminance correction coefficient of each pixelof images (a reference image and a comparison image) of the imaging unit100 a and the imaging unit 100 b. The correction coefficient is a valueat which the luminance of an image obtained by capturing an object withuniform light is the same in the entire image.

The geometric rectification information storage means 115 such as amemory or a hard disk stores a geometric rectification amount of eachpixel of images (a reference image and a comparison image) of theimaging unit 100 a and the imaging unit 100 b. The rectification amountis a value for rectification to an image in which distortions of theoptical element 101 a and the optical element 101 b, and errors of focallengths, errors of optical axis positions on images and mounting errorsof the imaging unit 100 a and the imaging unit 100 b are 0.

The synchronization signal transmission means 116 generates andtransmits a synchronization signal.

The reference image acquisition means 117 a sends a signal for openingthe shutter to the shutter means 102 a and acquires an image generatedby the imaging element 103 a, in synchronization with thesynchronization signal from the synchronization signal transmissionmeans 116.

The comparison image acquisition means 117 b sends a signal for openingthe shutter to the shutter means 102 b and acquires an image generatedby the imaging element 103 b, in synchronization with thesynchronization signal from the synchronization signal transmissionmeans 116.

The luminance correction means 118 reads the luminance correctioncoefficient of each pixel from the luminance correction informationstorage means 114 so as to correct luminance of a reference image and acomparison image.

The geometric rectification means 119 reads a two-dimensional geometricrectification amount of each pixel from the geometric rectificationinformation storage means 115 so as to geometrically rectify thereference image and the comparison image, thereby rectifying shapes ofreflected pictures.

The disparity calculation means 120 searches for a region of thecomparison image, corresponding to a region (template image) which has apredetermined size and is extracted from the reference image. Adifference between a position of the region on the comparison imagematching the template image and a position of the template image on thereference image, that is, disparity is calculated. Disparity iscalculated for each pixel, and thus a disparity image is calculated.

The distance calculation means 121 calculates a distance (range) fromthe imaging apparatus to an object on the images in the optical axisdirections of the imaging unit 100 a and the imaging unit 100 b on thebasis of the disparity calculated by the disparity calculation means120, distances (baseline lengths) of foci and focal lengths of theimaging unit 100 a and the imaging unit 100 b. The distance iscalculated for each pixel, and thus a distance image is calculated.

The recognition means 122 recognizes, the object reflected in thereference image and a position of the object on the reference image, andcalculates a three-dimensional relative position and relative speed ofthe object for the imaging apparatus, by using the reference image andthe distance image. Here, a three-dimensional relative positioncoordinate system of the imaging apparatus has an x coordinate in theright direction, a y coordinate in the upward direction, and a zcoordinate in the optical axis direction with respect to the imagingunit 100 a and the imaging unit 100 b, with a midpoint between the fociof the imaging unit 100 a and the imaging unit 100 b as an origin. Inaddition, on the basis of relative positions and relative speeds of theimaging apparatus and the object, time to collision is calculated, andit is determined whether or not the collision will occur within apredetermined time. The relative positions and the relative speeds ofthe imaging apparatus and the object, a collision determination result,and the time to collision are sent to the screen/sound output unit 130and the control unit 140.

The characteristic input/output means 123 acquires the informationregarding transmittance, distortion, and the like of the optical element101 a and the optical element 101 b, the information regarding asensitivity characteristic, noise, the number of defective pixels andthe like of the imaging element 103 a and the imaging element 103 b, orthe information regarding a sensitivity characteristic, noise, and thelike of the imaging unit 100 a and the imaging unit 100 b, stored in thecharacteristic storage means 104 a and the characteristic storage means104 b, and outputs the information to outside of the imaging apparatus.

The screen/sound output unit 130 such as a monitor and a speakerdisplays the reference image or the disparity image, and the distanceimage on the screen. In addition, a frame or a marker is displayed atthe position of the object. In this case, a color of a frame or a markerof an object of which a collision determination result from therecognition means 122 indicates collision is set to be different fromthat of an object of which a collision determination result isnon-collision. In a case where there is an object of which a collisiondetermination result from the recognition means 122 indicates collision,a warning sound is output.

The control unit 140 such as a CPU generates a control signal on thebasis of the relative positions and relative speeds of the imagingapparatus and the object, the collision time, and the collisiondetermination result, and outputs the control signal to the outside ofthe imaging apparatus.

With reference to FIG. 2, a description will be made of operationprocedures of the imaging apparatus according to the embodiment of thepresent invention illustrated in FIG. 1.

Step 201: The synchronization signal transmission means 116 generates asynchronization signal and sends the synchronization signal to thereference image acquisition means 117 a and the comparison imageacquisition means 117 b. The reference image acquisition means 117 asends a shutter opening/closing signal and exposure time information tothe shutter means 102 a right after receiving the synchronization signalfrom the synchronization signal transmission means 116. The shuttermeans 102 a opens the shutter mechanism only for the exposure time rightafter receiving the shutter opening/closing signal and the exposure timeinformation from the reference image acquisition means 117 a, and thencloses the shutter mechanism. The imaging element 103 a receives animage of light refracted by the optical element 101 a, generates animage corresponding to the intensity of the light, and sends the imageto the reference image acquisition means 117 a. The reference imageacquisition means 117 a receives the image from the imaging element 103a and stores the image in the reference image storage means 111.

The comparison image acquisition means 117 b sends a shutteropening/closing signal and exposure time information to the shuttermeans 102 b right after receiving the synchronization signal from thesynchronization signal transmission means 116. The shutter means 102 bopens the shutter mechanism only for the exposure time right afterreceiving the shutter opening/closing signal and the exposure timeinformation from the comparison image acquisition means 117 b, and thencloses the shutter mechanism. The imaging element 103 b receives animage of light refracted by the optical element 101 b, generates animage corresponding to the intensity of the light, and sends the imageto the comparison image acquisition means 117 b. The comparison imageacquisition means 117 b receives the image from the imaging element 103b and stores the image in the comparison image storage means 112.

Step 202: The luminance correction means 118 reads a correctioncoefficient of each pixel of the images generated by the imaging element103 a and the imaging element 103 b from the luminance correctioninformation storage means 114, and reads the reference image and thecomparison image from the reference image storage means 111 and thecomparison image storage means 112, respectively. A luminance value ofthe reference image is corrected by multiplying the correctioncoefficient of each pixel of the image generated by the imaging elementfor the reference image by a luminance value of each pixel of thereference image. Similarly, a luminance value of the comparison image iscorrected by multiplying the correction coefficient of each pixel of theimage generated by the imaging element for the comparison image by aluminance value of each pixel of the comparison image. The correctedreference image and comparison image are respectively stored in thereference image storage means 111 and the comparison image storage means112.

Step 203: The geometric rectification means 119 reads a two-dimensionalgeometric rectification amount of each pixel of the images generated bythe imaging element 103 a and the imaging element 103 b from thegeometric rectification information storage means 115, and reads thereference image and the comparison image from the reference imagestorage means 111 and the comparison image storage means 112,respectively. A position on the reference image of which thetwo-dimensional rectification coefficient is changed is calculated fromeach pixel of the reference image, and a luminance value of the positionis calculated from luminance values around the position throughinterpolation calculation. This calculation is performed on all pixelsof the reference image. Similarly, a position on the comparison image ofwhich the two-dimensional rectification coefficient is changed iscalculated from each pixel of the comparison image, and a luminancevalue of the position is calculated from luminance values around theposition through interpolation calculation. This calculation isperformed on all pixels of the comparison image. The rectified referenceimage and comparison image are respectively stored in the referenceimage storage means 111 and the comparison image storage means 112.

Step 204: The disparity calculation means 120 extracts an image 303(template image) of a region with a predetermined size on the referenceimage 301 as illustrated in FIG. 3. An image of a region in which thesame object as in the template image 303 is reflected is searched for onthe comparison image 302. An image 304 of a region with a predeterminedsize on the comparison image 302 is extracted; a sum of absolutedifferences (SAD) between luminance values of the template image 303 onthe reference image 301 and luminance values of the image 304 of theregion with the predetermined size on the comparison image 302 iscalculated for each image 304 of each region on the comparison image302; and a distance between the image 304 of the region with thesmallest value on the comparison image 302 and the region of thetemplate image 303, that is, disparity is calculated. This process isperformed on all regions on the reference image 301, so as to calculatedisparity for the entire reference image 301. A disparity imagecalculated in the above-described manner is stored in the processedimage storage means 113.

Step 205: The distance calculation means 121 reads the disparity imagefrom the processed image storage means 113. A value obtained bymultiplying a distance between the foci of the imaging unit 100 a andthe imaging unit 100 b by the focal lengths thereof is divided by thedisparity of each region, calculated in step 204, and a distance betweena picture reflected in the image 303 of the region on the referenceimage and the imaging apparatus in the optical axis direction iscalculated. This process is performed on all of the regions on thereference image, so that a distance between each picture and the imagingapparatus in the optical axis direction is calculated in the entirereference image. A distance image calculated in the above-describedmanner is stored in the processed image storage means 113.

Step 206: The recognition means 122 reads the reference image from thereference image storage means 111 and reads the distance image from theprocessed image storage means 113. Therefore, calculation of a positionof a vanishing position on the reference image, a determination of anobject such as an automobile or a pedestrian, calculation of a relativeposition and a relative speed of the object to the imaging apparatus,and a determination of collision between the object and the imagingapparatus are performed.

First, the recognition means 122 calculates a position of a vanishingpoint on the reference image in the following procedures. White lines ofboth sides located at lanes on the reference image are detected, and theslopes of the white lines on the reference image are calculated. Byusing the slopes calculated under the assumption that both of the whitelines are straight lines, a position of a point at which both of thewhite lines intersect each other on the reference image is calculated.This is a position of the vanishing point.

Next, the recognition means 122 detects an object such as an automobileor a pedestrian. In the distance image, a region connected by pixelswithin a predetermined distance range is obtained. As examples of thepredetermined range, there are 5 m to 10 m, 7.5 m to 12.5 m, and 10 m to15 m, and a plurality of ranges which overlap each other every 2.5 m ina width of 5 m are set. In relation to each region connected by thepixels within the predetermined distance range, vertical and horizontallengths of the region on the reference image are obtained. A value,obtained by multiplying the vertical length of each region on thereference image, the distance, and a pixel pitch by each other, isdivided by the focal length, so that a three-dimensional vertical lengthof each region is calculated. Similarly, a value, obtained bymultiplying the horizontal length of each region on the reference image,the distance, and the pixel pitch by each other, is divided by the focallength, so that a three-dimensional horizontal length of each region iscalculated.

By using Equation 1, a vertical position Vg of each region on thereference image with respect to the ground surface is approximatelycalculated. Here, Vv indicates a height of the vanishing point, findicates a focal length, Hi indicates a mounting height of the imagingapparatus, Lr indicates a mean distance of the region, and c indicates apixel pitch. In addition, Equation 1 is a computation expressionassuming that the optical axes of the imaging unit 100 a and the imagingunit 100 b are substantially present in the horizontal direction.

Vg=Vv−f×Hi/(Lr×c)  [Equation 1]

In a case where the three-dimensional vertical and horizontal lengths ofthe region are within predetermined ranges of an automobile, and adifference between a lower limit vertical position of the region on thereference image and a vertical position of the region on the referenceimage from the ground surface, calculated by using Equation 1, is withina threshold value, the object of the region is, determined as being anautomobile. Similarly, in a case where the three-dimensional verticaland horizontal lengths of the region are within predetermined ranges ofa pedestrian, and a difference between a lower limit vertical positionof the region on the reference image and a vertical position of theregion on the reference image from the ground surface, calculated byusing Equation 1, is within a threshold value, an object of the regionis determined as being a pedestrian. This process is performed on all ofthe regions, so that it is determined whether the object is anautomobile or a pedestrian.

Next, a relative position and a relative speed of the object to theimaging apparatus are calculated in the following procedures. Inrelation to the region determined as being the automobile or thepedestrian, relative positions (Xo, Yo, and Zo) of the object to theimaging apparatus is calculated by using Equation 2 to Equation 4. Here,(Uo and Vo) indicate a position of a center of region determined asbeing the automobile or the pedestrian on the reference image.

Xo=Lr×c×Uo/f  [Equation 2]

Yo=H+Lr×c×(Vo−Vv)/f  [Equation 3]

Zo=Lr  [Equation 4]

The processes of steps 201 to 208 are repeatedly performed in apredetermined cycle. In a case where a difference between the positionsof regions on the reference image, detected in the previous and presentprocesses of step 206 is within a predetermined value, the same objectis determined. A value obtained by subtracting the relative positioncalculated in the previous process of step 206 from the relativeposition of the object to the imaging apparatus, calculated in thepresent process, is divided by a time interval of the process cycle ofsteps 201 to 208, so that relative speeds (Vx, Vy, and Vz) of the objectto the imaging apparatus are calculated.

Finally, collision between the object and the imaging apparatus isdetermined in the following procedures. In a case where the relativespeed Vz of the object to the imaging apparatus is 0 or higher, it isdetermined that collision with the object of the region determined asbeing the automobile or the pedestrian will not occur. If the relativespeed Vz of the object to the imaging apparatus is a negative value, therelative position Zo of the object to the imaging apparatus, calculatedin the present process, is divided by the absolute value of the relativespeed Vz of the object to the imaging apparatus so that the time tocollision (collision time) is calculated. A value, obtained bymultiplying the collision time to the relative speed Vx of the object tothe imaging apparatus, is added to the relative position Xo of theobject, so that the relative position Xo of the object to the imagingapparatus at the time of the collision is calculated. Therefore, in acase where the relative speed Vz of the object to the imaging apparatusis a negative value, the collision time is within a threshold value, andan absolute value of the relative position Xo of the object to theimaging apparatus at the time of the collision is within a thresholdvalue, it is determined that collision with the object of the regiondetermined as the automobile or the pedestrian will occur. Otherwise, itis determined that collision will not occur. The recognition means 122sends positions of four corners of the region determined as being theautomobile or the pedestrian on the reference image, the relativepositions and the relative speeds of the object to the imagingapparatus, the collision determination result, and the collision time,to the screen/sound output unit 130 and the control unit 140.

Step 207: The screen/sound output unit 130 receives the positions offour corners of the region determined as being the automobile or thepedestrian on the reference image, the relative positions and therelative speeds of the object to the imaging apparatus, the collisiondetermination result, and the collision time, from the recognition means122. The reference image is read from the reference image storage means111. The reference image is displayed on a screen, and the regiondetermined as being the automobile or the pedestrian is displayed as aframe. In addition, a color of a frame of a region of which a collisiondetermination result indicates collision is displayed on the screen soat to be changed to a color of a frame of a region of an object of whicha collision determination result indicates non-collision. If there is acollision determination result indicating collision in the region, awarning sound is output.

Step 208: The control unit 140 receives the positions of four corners ofthe region determined as being the automobile or the pedestrian on thereference image, the relative positions and the relative speeds of theobject to the imaging apparatus, the collision determination result, andthe collision time, from the recognition means 122. If there is acollision determination result indicating collision in the regiondetermined as being the automobile or the pedestrian will occur, acontrol signal for avoiding the collision is output to the outside ofthe imaging apparatus.

A description will be made of operation procedures of the imagingapparatus according to the embodiment of the present inventionillustrated in FIG. 1.

The characteristic input/output means 123 reads, from the characteristicstorage means 104 a and the characteristic storage means 104 b, theinformation regarding transmittance and distortions (a distortioncoefficient of a lens in a radial direction, a distortion coefficient ofa lens in a tangential direction, and the like) of the optical element101 a and the optical element 101 b; the information regardingsensitivity characteristics (a luminance value of an image obtained byimaging an object with uniform light, a dynamic range, and the like),noise (an SN ratio, a standard deviation (variation) of luminance valuesof an image obtained by imaging an object with uniform light, shot noiseof light with predetermined intensity, dark current noise, readingnoise, fixed pattern noise of light with predetermined intensity, andthe like), the number of defective pixels, and the like of the imagingelement 103 a and the imaging element 103 b; and the informationregarding sensitivity characteristics (dynamic ranges of the imagingelement 103 a and the imaging element 103 b which receive light havingpassed through the optical element 101 a and the optical element 101 b,luminance values of images of the imaging element 103 a and the imagingelement 103 b which receive light having passed through the opticalelement 101 a and the optical element 101 b when an object is imagedwith uniform light, and the like), noise (SN ratios of the imagingelement 103 a and the imaging element 103 b which receive light havingpassed through the optical element 101 a and the optical element 101 b,standard deviations (variations) of luminance values of images of theimaging element 103 a and the imaging element 103 b which receive lighthaving passed through the optical element 101 a and the optical element101 b when an object is imaged with uniform light, shot noise of theimaging element 103 a and the imaging element 103 b which receive lighthaving passed through the optical element 101 a and the optical element101 b when light with predetermined intensity is incident, dark currentnoise of the imaging element 103 a and the imaging element 103 b whichreceive light having passed through the optical element 101 a and theoptical element 101 b, reading noise of the imaging element 103 a andthe imaging element 103 b which receive light having passed through theoptical element 101 a and the optical element 101 b, fixed pattern noiseof the imaging element 103 a and the imaging element 103 b which receivelight having passed through the optical element 101 a and the opticalelement 101 b when light with predetermined intensity is incident, andthe like), and the like of the imaging unit 100 a and the imaging unit100 b, and outputs the information to the outside of the imagingapparatus.

According to the operation procedures (FIG. 2) of the imaging apparatusof the embodiment of the present invention illustrated in FIG. 1, theimaging unit 100 a for a reference image and the imaging unit 100 binclude the optical element 101 a and the optical element 101 b or theimaging element 103 a and the imaging element 103 b, satisfying one itema among the following items 1-1 to 1-7 which are set in advance.

For this reason, compared with a case of not satisfying the item a,quality of a reference image becomes higher than that of a comparisonimage, and an object recognition process is performed in step 206 byusing the reference image with the higher quality, so that objectrecognition performance is improved. In addition, if an imagingapparatus is manufactured without taking the item a into consideration,a case of not satisfying the item a occurs. If an imaging apparatus ismanufactured so as to satisfy the item a, object recognition performanceis improved in this case compared with an imaging apparatus which ismanufactured without taking the item a into consideration, and thus itis possible to reduce variations in the object recognition performancefor each of imaging apparatuses.

In addition, in step 207, the screen/sound output unit 130 displays aframe with a predetermined color in an object with which it isdetermined that “collision” will occur on the reference image of thescreen, and outputs a warning sound. Therefore, compared with a case ofnot satisfying the item a, the object recognition performance isimproved, and thus it is possible to notify a user of the collidingobject more rapidly and more reliably.

In addition, in step S208, if there is an object with which it isdetermined that “collision” will occur on the reference image of thescreen, the control unit generates a control signal for avoiding thecollision and outputs to outside of the imaging apparatus. Therefore,compared with a case of not satisfying the item a, the objectrecognition performance is improved, and therefore it is possible toperform control for avoiding the object more rapidly and more reliablyand thus to reduce a possibility of the collision.

-   -   Item 1-1: The transmittance of the optical element 101 a for a        reference image is higher than that of the optical element 101 b        for a comparison image.    -   Item 1-2: The distortion of the optical element 101 a for a        reference image is smaller than that of the optical element 101        b for a comparison image.    -   Item 1-3: The sensitivity characteristic of the imaging element        103 a for a reference image is higher than that of the imaging        element 103 b for a comparison image.    -   Item 1-4: The level of noise of the imaging element 103 a for a        reference image is lower than that of the imaging element 103 b        for a comparison image.    -   Item 1-5: The number of defective pixels of the imaging element        103 a for a reference image is smaller than that of the imaging        element 103 b for a comparison image.    -   Item 1-6: The sensitivity characteristic of the imaging unit 100        a for a reference image is higher than that of the imaging unit        100 b for a comparison image.    -   Item 1-7: The level of noise of the imaging unit 100 a for a        reference image is lower than that of the imaging unit 100 b for        a comparison image.

According to the operation procedures of the imaging apparatus of theembodiment of the present invention illustrated in FIG. 1, thecharacteristic input/output means 123 reads the information regardingtransmittance, distortions, and the like of the optical element 101 aand the optical element 101 b, the information regarding sensitivitycharacteristics, noise, the number of defective pixels, and the like ofthe imaging element 103 a and the imaging element 103 b, and theinformation regarding sensitivity characteristics, noise, and the likeof the imaging unit 100 a and the imaging unit 100 b, stored in thecharacteristic storage means 104 a and the characteristic storage means104 b, and outputs the information to outside of the imaging apparatus.Therefore, on the basis of the information values, it is possible tocheck whether or not one of the above-described items 1-1 to 1-7 issatisfied.

According to the operation procedures (FIG. 2) of the imaging apparatusof the embodiment of the present invention illustrated in FIG. 1, instep 207, the screen/sound output unit 130 displays a frame with apredetermined color in an object with which it is determined that“collision” will occur on the reference image of the screen, and thus itis possible to notify a user of the colliding object.

Further, the imaging apparatus of the present invention is not limitedto the above-described embodiment, and may be applied through variousmodifications. Hereinafter, modification examples of the imagingapparatus of the present invention will be described.

Modification Example 1-1

In the embodiment of the imaging apparatus of the present inventionillustrated in FIG. 1, also in a case where the imaging unit 100 a andthe imaging unit 100 b include the optical element 101 a and the opticalelement 101 b or the imaging element 103 a and the imaging element 103b, satisfying one item b among the following items 1-11 to 1-30 whichare predetermined conditions set in advance, compared with a case of notsatisfying the item b, quality of a reference image becomes higher thanthat of a comparison image, and an object recognition process isperformed in step 206 by using the reference image with the higherquality, so that object recognition performance is improved.

In addition, if an imaging apparatus is manufactured without taking theitem b into consideration, a case of not satisfying the item b occurs.If an imaging apparatus is manufactured so as to satisfy the item b,object recognition performance is improved in this case compared with animaging apparatus which is manufactured without taking the item b intoconsideration, and thus it is possible to reduce variations in theobject recognition performance for each of imaging apparatuses.

In addition, in step 207, the screen/sound output unit 130 displays aframe with a predetermined color in an object with which it isdetermined that “collision” will occur on the reference image of thescreen, and outputs a warning sound. Therefore, compared with a case ofnot satisfying the item b, the object recognition performance isimproved, and thus it is possible to notify a user of the collidingobject more rapidly and more reliably.

In addition, if there is an object with which it is determined that“collision” will occur on the reference image of the screen in step 208,the control unit generates a control signal for avoiding the collisionand outputs to outside of the imaging apparatus. Therefore, comparedwith a case of not satisfying the item b, the object recognitionperformance is improved, and therefore it is possible to perform controlfor avoiding the object more rapidly and more reliably and thus toreduce a possibility of the collision.

-   -   Item 1-11: The transmittance of the optical element 101 a is        higher than that of the optical element 101 b.    -   Item 1-12: The distortion coefficient of the lens in the radial        direction of the optical element 101 a is smaller than that of        the optical element 101 b.    -   Item 1-13: The distortion coefficient of the lens in the        tangential direction of the optical element 101 a is smaller        than that of the optical element 101 b.    -   Item 1-14: The dynamic range of the imaging element 103 a is        wider than that of the imaging element 103 b.    -   Item 1-15: The luminance value of an image in uniform light of        the imaging element 103 a is greater than that of the imaging        element 103 b.    -   Item 1-16: The SN ratio of the imaging element 103 a is smaller        than that of the imaging element 103 b.    -   Item 1-17: The standard deviation of luminance values of an        image in uniform light of the imaging element 103 a is smaller        than that of the imaging element 103 b.    -   Item 1-18: The level of shot noise of light with predetermined        intensity of the imaging element 103 a is lower than that of the        imaging element 103 b.    -   Item 1-19: The level of dark current noise of the imaging        element 103 a is lower than that of the imaging element 103 b.    -   Item 1-20: The level of reading noise of the imaging element 103        a is lower than that of the imaging element 103 b.    -   Item 1-21: The level of fixed pattern noise with predetermined        intensity of the imaging element 103 a is lower than that of the        imaging element 103 b.    -   Item 1-22: The number of defective pixels of the imaging element        103 a is lower than that of the imaging element 103 b.    -   Item 1-23: The dynamic range of the imaging element 103 a which        receives light having passed through the optical element 101 a        is wider than that of the imaging element 103 b which receives        light having passed through the optical element 101 b.    -   Item 1-24: The luminance value of an image in uniform light of        the imaging element 103 a which receives light having passed        through the optical element 101 a is greater than that of the        imaging element 103 b which receives light having passed through        the optical element 101 b.    -   Item 1-25: The SN ratio of the imaging element 103 a which        receives light having passed through the optical element 101 a        is smaller than that of the imaging element 103 b which receives        light having passed through the optical element 101 b.    -   Item 1-26: The standard deviation of luminance values of an        image in uniform light of the imaging element 103 a which        receives light having passed through the optical element 101 a        is smaller than that of the imaging element 103 b which receives        light having passed through the optical element. 101 b.    -   Item 1-27: The level of shot noise of light with predetermined        intensity of the imaging element 103 a which receives light        having passed through the optical element 101 a is lower than        that of the imaging element 103 b which receives light having        passed through the optical element 101 b.    -   Item 1-28: The level of dark current noise of the imaging        element 103 a which receives light having passed through the        optical element 101 a is lower than that of the imaging element        103 b which receives light having passed through the optical        element 101 b.    -   Item 1-29: The level of reading noise of the imaging element 103        a which receives light having passed through the optical element        101 a is lower than that of the imaging element 103 b which        receives light having passed through the optical element 101 b.    -   Item 1-30: The level of fixed pattern noise of light with        predetermined intensity of the imaging element 103 a which        receives light having passed through the optical element 101 a        is lower than that of the imaging element 103 b which receives        light having passed through the optical element 101 b.

In operation procedures of the imaging apparatus according to theembodiment, of the present invention illustrated in FIG. 1, thecharacteristic input/output means 123 reads, from the characteristicstorage means 104 a and the characteristic storage means 104 b, theinformation regarding transmittance, and distortion coefficients of thelenses in the radial direction and the tangential direction of theoptical element. 101 a and the optical element 101 b; the informationregarding dynamic ranges, luminance values of images obtained by imagingan object with uniform light, SN ratios, standard deviations(variations) of luminance values of images obtained by imaging an objectwith uniform light, shot noise of light with predetermined intensity,dark current noise, reading noise, fixed pattern noise of light withpredetermined intensity, and the number of defective pixels of theimaging element 103 a and the imaging element 103 b; and dynamic rangesof the imaging element 103 a and the imaging element 103 b which receivelight having passed through the optical element 101 a and the opticalelement 101 b, luminance values of images of the imaging element 103 aand the imaging element 103 b which receive light having passed throughthe optical element 101 a and the optical element 101 b when an objectis imaged with uniform light, SN ratios of the imaging element 103 a andthe imaging element 103 b which receive light having passed through theoptical element 101 a and the optical element 101 b, standard deviations(variations) of luminance values of images of the imaging element 103 aand the imaging element 103 b which receive light having passed throughthe optical element 101 a and the optical element 101 b when an objectis imaged with uniform light, shot noise of the imaging element 103 aand the imaging element 103 b which receive light having passed throughthe optical element 101 a and the optical element 101 b when light withpredetermined intensity is incident, dark current noise of the imagingelement 103 a and the imaging element 103 b which receive light havingpassed through the optical element 101 a and the optical element 101 b,reading noise of the imaging element 103 a and the imaging element 103 bwhich receive light having passed through the optical element 101 a andthe optical element 101 b, fixed pattern noise of the imaging element103 a and the imaging element 103 b which receive light having passedthrough the optical element 101 a and the optical element 101 b whenlight with predetermined intensity is incident, and the like, andoutputs the information to outside of the imaging apparatus. Therefore,on the basis of the information values, it is possible to check whetheror not one of the above-described items 1-11 to 1-30 is satisfied.

Modification Example 1-2

In step 204 of the operation procedures (FIG. 2) of the imagingapparatus according to the embodiment of the present inventionillustrated in FIG. 1, also in a case where the disparity calculationmeans 120 calculates a value of an SAD, and searches for the smallestregion on the comparison image so as to calculate disparity.Alternatively, the disparity calculation means may calculate a zero-meansum of absolute differences (ZSAD), a sum of squared differences (SSD),a zero-mean sum of squared differences (ZSSD), normalized crosscorrelation (NCC), or zero-mean cross correlation (ZNCC) and searchesfor the smallest region on the comparison image so as to calculatedisparity, it is possible to obtain the disparity.

Modification Example 1-3

In the imaging apparatus according to the embodiment of the presentinvention illustrated in FIG. 1, also in a case where the imagingelement 103 a and the imaging element 103 b are respectively providedwith imaging element characteristic storage means 105 a and imagingelement characteristic storage means 105 b such as memories which storeinformation regarding sensitivity characteristics, noise, the number ofdefective pixels, and the like of the imaging element 103 a and theimaging element 103 b; and in the operation procedures of the imagingapparatus according to the embodiment of the present inventionillustrated in FIG. 1, the characteristic input/output means 123 readsthe information regarding sensitivity characteristics, noise, the numberof defective pixels, and the like of the imaging element 103 a and theimaging element 103 b, from the imaging element characteristic storagemeans 105 a and the imaging element characteristic storage means 105 b,and may output the information to outside of the imaging apparatus, onthe basis of the information values, it is possible to check whether ornot one of the items 1-3 to 1-5 is satisfied.

In addition, in operation procedures of the imaging apparatus accordingto the embodiment of the present invention illustrated in FIG. 1, alsoin a case where the characteristic input/output means 123 reads, fromthe imaging element characteristic storage means 105 a and the imagingelement characteristic storage means 105 b, the information regardingdynamic ranges, luminance values of images obtained by imaging an objectwith uniform light, SN ratios, standard deviations (variations) ofluminance values of images obtained by imaging an object with uniformlight, shot noise of light with predetermined intensity, dark currentnoise, reading noise, fixed pattern noise of light with predeterminedintensity, and the number of defective pixels of the imaging element 103a and the imaging element 103 b, and outputs the information to outsideof the imaging apparatus, on the basis of the information values, it ispossible to check whether or not one of the above-described items 1-14to 1-22 is satisfied.

Modification Example 1-4

In the imaging apparatus according to the embodiment of the presentinvention illustrated in FIG. 1, also in a case where the characteristicstorage means 104 a and the characteristic storage means 104 b areprovided not in the imaging unit 100 a and the imaging unit 100 b but inthe calculation unit 110; and in operation procedures of the imagingapparatus according to the embodiment of the present inventionillustrated in FIG. 1, the characteristic input/output means 123 readsthe information regarding transmittance, distortions, and the like ofthe optical element 101 a and the optical element 101 b, the informationregarding sensitivity characteristics, noise, the number of defectivepixels, and the like of the imaging element 103 a and the imagingelement 103 b, and the information regarding sensitivitycharacteristics, noise, and the like of the imaging unit 100 a and theimaging unit 100 b, stored in the characteristic storage means 104 a andthe characteristic storage means 104 b, and outputs the information tooutside of the imaging apparatus, on the basis of the informationvalues, it is possible to check whether or not one of theabove-described items 1-1 to 1-7 is satisfied.

In addition, in operation procedures of the imaging apparatus accordingto the embodiment of the present invention illustrated in FIG. 1, alsoin a case where the characteristic input/output means 123 reads, fromthe characteristic storage means 104 a and the characteristic storagemeans 104 b, the information regarding transmittance, and distortioncoefficients of the lenses in the radial direction and the tangentialdirection of the optical element 101 a and the optical element 101 b;the information regarding dynamic ranges, luminance values of imagesobtained by imaging an object with uniform light, SN ratios, standarddeviations (variations) of luminance values of images obtained byimaging an object with uniform light, shot noise of light withpredetermined intensity, dark current noise, reading noise, fixedpattern noise of light with predetermined intensity, and the number ofdefective pixels of the imaging element 103 a and the imaging element103 b; and dynamic ranges of the imaging element 103 a and the imagingelement 103 b which receive light having passed through the opticalelement 101 a and the optical element 101 b, luminance values of imagesof the imaging element 103 a and the imaging element 103 b which receivelight having passed through the optical element 101 a and the opticalelement 101 b when an object is imaged with uniform light, SN ratios ofthe imaging element 103 a and the imaging element 103 b which receivelight having passed through the optical element 101 a and the opticalelement 101 b, standard deviations (variations) of luminance values ofimages of the imaging element 103 a and the imaging element 103 b whichreceive light having passed through the optical element 101 a and theoptical element 101 b when an object is imaged with uniform light, shotnoise of the imaging element 103 a and the imaging element 103 b whichreceive light having passed through the optical element 101 a and theoptical element 101 b when light with predetermined intensity isincident, dark current noise of the imaging element 103 a and theimaging element 103 b which receive light having passed through theoptical element 101 a and the optical element 101 b, reading noise ofthe imaging element 103 a and the imaging element 103 b which receivelight having passed through the optical element 101 a and the opticalelement 101 b, fixed pattern noise of the imaging element 103 a and theimaging element 103 b which receive light having passed through theoptical element 101 a and the optical element 101 b when light withpredetermined intensity is incident, and the like, and outputs theinformation to outside of the imaging apparatus, on the basis of theinformation values, it is possible to check whether or not one of theabove-described items 1-11 to 1-30 is satisfied.

Modification Example 1-5

In step 207 of the operation procedures (FIG. 2) of the imagingapparatus according to the embodiment of the present inventionillustrated in FIG. 1, also in a case where the screen/sound output unit130 displays the disparity image or the distance image instead of thereference image on the screen and display a frame with a predeterminedcolor in an object with which it is determined that “collision” willoccur, it is possible to notify a user of the colliding object.

Modification Example 1-6

In the imaging apparatus according to the embodiment of the presentinvention illustrated in FIG. 1, not only two imaging units but alsothree or more imaging units are provided, and the operation proceduresillustrated in FIG. 2 are performed on a combination of two of theplurality of imaging units. Therefore, a distance to an object imaged bythe plurality of imaging units or the object can be recognized, and, inthe combination of the two of the plurality of imaging units, an imagingunit for a reference image includes an optical element or an imagingelement satisfying one item a among the preset items 1-1 to 1-7.

For this reason, compared with a case of not satisfying the item a,quality of a reference image becomes higher than that of a comparisonimage, and an object recognition process is performed in step 206 byusing the reference image with the higher quality, so that objectrecognition performance is improved.

In addition, if an imaging apparatus is manufactured without taking theitem a into consideration, a case of not satisfying the item a occurs.If an imaging apparatus is manufactured so as to satisfy the item a,object recognition performance is improved in this case compared with animaging apparatus which is manufactured without taking the item a intoconsideration, and thus it is possible to reduce variations in theobject recognition performance for each of imaging apparatuses.

In addition, in step 207, the screen/sound output unit 130 displays aframe with a predetermined color in an object with which it isdetermined that “collision” will occur on the reference image of thescreen, and outputs a warning sound. Therefore, compared with a case ofnot satisfying the item a, the object recognition performance isimproved, and thus it is possible to notify a user of the collidingobject more rapidly and more reliably.

In addition, in step S208, if there is an object with which it isdetermined that “collision” will occur on the reference image of thescreen, the control unit generates a control signal for avoiding thecollision and outputs to outside of the imaging apparatus. Therefore,compared with a case of not satisfying the item a, the objectrecognition performance is improved, and therefore it is possible toperform control for avoiding the object more rapidly and more reliablyand thus to reduce a possibility of the collision.

Embodiment 2

FIG. 4 illustrates a configuration of an imaging apparatus according toanother embodiment of the present invention.

The imaging apparatus according to the embodiment of the presentinvention includes an imaging unit 400 a, an imaging unit 400 b, acalculation unit 410, a screen/sound output unit 130, and a control unit140. The screen/sound output unit 130 and the control unit 140 are thesame as those illustrated in FIG. 1, and thus description thereof willbe omitted.

The imaging unit 400 a such as a camera includes an optical element 101a, shutter means 102 a, and an imaging element 103 a.

The optical element 101 a, the shutter means 102 a, and the imagingelement 103 a are the same as those illustrated in FIG. 1, anddescription thereof will be omitted.

The imaging unit 400 b such as a camera includes an optical element 101b, shutter means 102 b, and an imaging element 103 b.

The optical element 101 b, the shutter means 102 b, and the imagingelement 103 b are the same as those illustrated in FIG. 1, anddescription thereof will be omitted.

The calculation unit 410 constituted by a central processing unit (CPU),a memory, and the like includes reference image storage means 111,comparison image storage means 112, processed image storage means 113,characteristic storage means 404, luminance correction informationstorage means 114, geometric rectification information storage means115, synchronization signal transmission means 116, image acquisitionmeans 417 a, image acquisition means 417 b, reference image selectionmeans 424, luminance correction means 118, geometric rectification means119, disparity calculation means 120, distance calculation means 121,recognition means 122, and characteristic input/output means 423.

The reference image storage means 111, the comparison image storagemeans 112, the processed image storage means 113, the luminancecorrection information storage means 114, the geometric rectificationinformation storage means 115, the synchronization signal transmissionmeans 116, the luminance correction means 118, the geometricrectification means 119, the disparity calculation means 120, thedistance calculation means 121, and the recognition means 122 are thesame as those illustrated in FIG. 1, and description thereof will beomitted.

The characteristic storage means 404 such as a memory and a hard diskstores information regarding transmittance, distortion, and the like ofthe optical element 101 a and the optical element 101 b, informationregarding sensitivity characteristics, noise, the number of defectivepixels, and the like of the imaging element 103 a and the imagingelement 103 b, and information regarding sensitivity characteristics,noise, and the like of the imaging unit 400 a and the imaging unit 400b. The information regarding distortion of the optical element 101 a andthe optical element 101 b includes distortion coefficients of lenses ina radial direction, distortion coefficients of lenses in a tangentialdirection, and the like. The information regarding sensitivitycharacteristics of the imaging element 103 a and the imaging element 103b includes dynamic ranges, luminance values of images obtained byimaging an object with uniform light, and the like. The informationregarding noise of the imaging element 103 a and the imaging element 103b includes SN ratios, standard deviations (variations) of luminancevalues of images obtained by imaging an object with uniform light, shotnoise of light with predetermined intensity, dark current noise, readingnoise, fixed pattern noise of light with predetermined intensity, andthe like. The information regarding a sensitivity characteristic of theimaging unit 400 a and the imaging unit 400 b includes dynamic range ofthe imaging element 103 a and the imaging element 103 b which receivelight having passed through the optical element 101 a and the opticalelement 101 b, luminance values of images of the imaging element 103 aand the imaging element 103 b which receive light having passed throughthe optical element 101 a and the optical element 101 b when an objectis imaged with uniform light, and the like. The information regardingnoise of the imaging unit 400 a and the imaging unit 400 b includes SNratios of the imaging element 103 a and the imaging element 103 b whichreceive light having passed through the optical element 101 a and theoptical element 101 b, standard deviations (variations) of luminancevalues of images of the imaging element 103 a and the imaging element103 b which receive light having passed through the optical element 101a and the optical element 101 b when an object is imaged with uniformlight, shot noise of the imaging element 103 a and the imaging element103 b which receive light having passed through the optical element 101a and the optical element 101 b when light with predetermined intensityis incident, dark current noise of the imaging element 103 a and theimaging element 103 b which receive light having passed through theoptical element 101 a and the optical element 101 b, reading noise ofthe imaging element 103 a and the imaging element 103 b which receivelight having passed through the optical element 101 a and the opticalelement 101 b, fixed pattern noise of the imaging element 103 a and theimaging element 103 b which receive light having passed through theoptical element 101 a and the optical element 101 b when light withpredetermined intensity is incident, and the like.

The image acquisition means 417 a sends a signal for opening the shutterto the shutter means 102 a and acquires an image generated by theimaging element 103 a, in synchronization with the synchronizationsignal from the synchronization signal transmission means 116.

The image acquisition means 417 b sends a signal for opening the shutterto the shutter means 102 b and acquires an image generated by theimaging element 103 b, in synchronization with the synchronizationsignal from the synchronization signal transmission means 116.

The reference image selection means 424 reads, from the characteristicstorage means 404, the information regarding transmittance, distortions,and the like of the optical element 101 a and the optical element 101 b,the information regarding sensitivity characteristics, noise, the numberof defective pixels, and the like of the imaging element 103 a and theimaging element 103 b, and the information regarding sensitivitycharacteristics, noise, and the like of the imaging unit 400 a and theimaging unit 400 b. The image acquisition means 417 a and the imageacquisition means 417 b respectively receive acquired images. An imageof the optical element or the imaging element satisfying one item camong the following items 2-1 to 2-7 which are predetermined conditionsset in advance is used as a reference image, and the other image is usedas a comparison image. The reference image is stored in the referenceimage storage means 111, and the comparison image is stored in thecomparison image storage means 112.

-   -   Item 2-1: The transmittance of the optical element is high.    -   Item 2-2: The distortion of the optical element is small.    -   Item 2-3: The sensitivity characteristic of the imaging element        is high.    -   Item 2-4: The level of noise of the imaging element is low.    -   Item 2-5: The number of defective pixels of the imaging element        is small.    -   Item 2-6: The sensitivity characteristic of the imaging unit is        high.    -   Item 2-7: The level of noise of the imaging unit is low.

The characteristic input/output means 423 acquires the informationregarding transmittance, distortion, and the like of the optical element101 a and the optical element 101 b, the information regarding asensitivity characteristic, noise, the number of defective pixels andthe like of the imaging element 103 a and the imaging element 103 b, orthe information regarding a sensitivity characteristic, noise, and thelike of the imaging unit 400 a and the imaging unit 400 b, stored in thecharacteristic storage means 404, and outputs the information to outsideof the imaging apparatus.

With reference to FIG. 5, a description will be made of operationprocedures of the imaging apparatus according to the embodiment of thepresent invention illustrated in FIG. 4. Here, processes of steps 202 to208 are the same as steps 202 to 208 of FIG. 2, and thus descriptionthereof will be omitted.

Step 501: The reference image selection means 424 reads the informationregarding transmittance, distortion, and the like of the optical element101 a and the optical element 101 b, the information regarding asensitivity characteristic, noise, the number of defective pixels andthe like of the imaging element 103 a and the imaging element 103 b, orthe information regarding a sensitivity characteristic, noise, and thelike of the imaging unit 400 a and the imaging unit 400 b, from thecharacteristic storage means 404. The image acquisition means 417 a andthe image acquisition means 417 b respectively receive acquired images.The imaging unit having the optical element or the imaging elementsatisfying one item c of the above-described preset items 2-1 to 2-7 isdetermined as an imaging unit for a reference image.

Step 502: The synchronization signal transmission means 116 generates asynchronization signal and sends the synchronization signal to the imageacquisition means 417 a and the image acquisition means 417 b. The imageacquisition means 417 a sends a shutter opening/closing signal andexposure time information to the shutter means 102 a right afterreceiving the synchronization signal from the synchronization signaltransmission means 116. The shutter means 102 a opens the shuttermechanism only for the exposure time right after receiving the shutteropening/closing signal and the exposure time information from the imageacquisition means 417 a, and then closes the shutter mechanism. Theimaging element 103 a receives an image of light refracted by theoptical element 101 a, generates an image corresponding to the intensityof the light, and sends the image to the image acquisition means 417 a.The image acquisition means 417 a receives the image from the imagingelement 103 a and sends the image to the reference image selection means424.

The image acquisition means 417 b sends a shutter opening/closing signaland exposure time information to the shutter means 102 b right afterreceiving the synchronization signal from the synchronization signaltransmission means 116. The shutter means 102 b opens the shuttermechanism only for the exposure time right after receiving the shutteropening/closing signal and the exposure time information from the imageacquisition means 417 b, and then closes the shutter mechanism. Theimaging element 103 b receives an image of light refracted by theoptical element 101 b, generates an image corresponding to the intensityof the light, and sends the image to the image acquisition means 417 b.The image acquisition means 417 b receives the image from the imagingelement 103 b and sends the image to the reference image selection means424.

Step 503: The reference image selection means 424 receives the imagesfrom the image acquisition means 417 a and the image acquisition means417 b, respectively. The image of the imaging unit for a referenceimage, determined in step 501, is used as a reference image, and theother image is used as a comparison image. The reference image is storedin the reference image storage means 111, and the comparison image isstored in the comparison image storage means 112.

A description will be made of operation procedures of the imagingapparatus according to the embodiment of the present inventionillustrated in FIG. 4.

The characteristic input/output means 423 reads, from the characteristicstorage means 404, the information regarding transmittance anddistortions (a distortion coefficient of a lens in a radial direction, adistortion coefficient of a lens in a tangential direction, and thelike) of the optical element 101 a and the optical element 101 b; theinformation regarding sensitivity characteristics (a dynamic range, aluminance value of an image obtained by imaging an object with uniformlight, and the like), noise (an SN ratio, a standard deviation(variation) of luminance values of an image obtained by imaging anobject with uniform light, shot noise of light with predeterminedintensity, dark current noise, reading noise, fixed pattern noise oflight with predetermined intensity, and the like), the number ofdefective pixels, and the like of the imaging element 103 a and theimaging element 103 b; and the information regarding sensitivitycharacteristics (dynamic ranges of the imaging element 103 a and theimaging element 103 b which receive light having passed through theoptical element 101 a and the optical element 101 b, luminance values ofimages of the imaging element 103 a and the imaging element 103 b whichreceive light having passed through the optical element 101 a and theoptical element 101 b when an object is imaged with uniform light, andthe like), noise (SN ratios of the imaging element 103 a and the imagingelement 103 b which receive light having passed through the opticalelement 101 a and the optical element 101 b, standard deviations(variations) of luminance values of images of the imaging element 103 aand the imaging element 103 b which receive light having passed throughthe optical element 101 a and the optical element 101 b when an objectis imaged with uniform light, shot noise of the imaging element 103 aand the imaging element 103 b which receive light having passed throughthe optical element 101 a and the optical element 101 b when light withpredetermined intensity is incident, dark current noise of the imagingelement 103 a and the imaging element 103 b which receive light havingpassed through the optical element 101 a and the optical element 101 b,reading noise of the imaging element 103 a and the imaging element 103 bwhich receive light having passed through the optical element 101 a andthe optical element 101 b, fixed pattern noise of the imaging element103 a and the imaging element 103 b which receive light having passedthrough the optical element 101 a and the optical element 101 b whenlight with predetermined intensity is incident, and the like), and thelike of the imaging unit 400 a and the imaging unit 400 b, and outputsthe information to the outside of the imaging apparatus.

According to the operation procedures (FIG. 5) of the imaging apparatusaccording to the embodiment of the present invention illustrated in FIG.4, in step 501, the reference image selection means 424 sets the imagingunit having the optical element or the imaging element satisfying oneitem c of the above-described preset items 2-1 to 2-7 as an imaging unitfor a reference image, and, in step 503, the reference image selectionmeans 424 sets the image of the imaging unit for a reference image, setin step 501, as a reference image and sets the other image as acomparison image. Therefore, compared with a case of not satisfying theitem c, quality of a reference image becomes higher than that of acomparison image, and an object recognition process is performed in step206 by using the reference image with the higher quality, so that objectrecognition performance is improved.

In addition, if an imaging apparatus is manufactured without taking theitem c into consideration, a case of not satisfying the item c occurs.If an imaging apparatus is manufactured so as to satisfy the item c,object recognition performance is improved in this case compared with animaging apparatus which is manufactured without taking the item c intoconsideration, and thus it is possible to reduce variations in theobject recognition performance for each of imaging apparatuses.

Further, in step 207, the screen/sound output unit 130 displays a framewith a predetermined color in an object with which it is determined that“collision” will occur on the reference image of the screen, and outputsa warning sound. Therefore, compared with a case of not satisfying theitem c, the object recognition performance is improved, and thus it ispossible to notify a user of the colliding object more rapidly and morereliably.

In addition, in step S208, if there is an object with which it isdetermined that “collision” will occur on the reference image of thescreen, the control unit generates a control signal for avoiding thecollision and outputs to outside of the imaging apparatus. Therefore,compared with a case of not satisfying the item c, the objectrecognition performance is improved, and therefore it is possible toperform control for avoiding the object more rapidly and more reliablyand thus to reduce a possibility of the collision.

According to the operation procedures of the imaging apparatus of theembodiment of the present invention illustrated in FIG. 4, thecharacteristic input/output means 423 reads the information regardingtransmittance, distortions, and the like of the optical element 101 aand the optical element 101 b, the information regarding sensitivitycharacteristics, noise, the number of defective pixels, and the like ofthe imaging element 103 a and the imaging element 103 b, and theinformation regarding sensitivity characteristics, noise, and the likeof the imaging unit 400 a and the imaging unit 400 b, stored in thecharacteristic storage means 404, and outputs the information to outsideof the imaging apparatus. Therefore, on the basis of the informationvalues, it is possible to check whether or not one of theabove-described items 2-1 to 2-7 is satisfied.

Further, the imaging apparatus of the present invention is not limitedto the above-described embodiment, and may be applied through variousmodifications. Hereinafter, modification examples of the imagingapparatus of the present invention will be described.

Modification Example 2-1

In step 501 of the embodiment of the imaging apparatus of the presentinvention illustrated in FIG. 4, also in a case where the referenceimage selection means 424 sets the imaging unit having the opticalelement or the imaging element satisfying one item d among the followingitems 2-11 to 2-30 which are predetermined conditions set in advance,instead of the above-described preset items 2-1 to 2-7, as an imagingunit for a reference image, compared with a case of not satisfying theitem d, quality of a reference image becomes higher than that of acomparison image, and an object recognition process is performed in step206 by using the reference image with the higher quality, so that objectrecognition performance is improved.

In addition, if an imaging apparatus is manufactured without taking theitem d into consideration, a case of not satisfying the item d occurs.If an imaging apparatus is manufactured so as to satisfy the item d,object recognition performance is improved in this case compared with animaging apparatus which is manufactured without taking the item d intoconsideration, and thus it is possible to reduce variations in theobject recognition performance for each of imaging apparatuses.

In addition, in step 207, the screen/sound output unit 130 displays aframe with a predetermined color in an object with which it isdetermined that “collision” will occur on the reference image of thescreen, and outputs a warning sound. Therefore, compared with a case ofnot satisfying the item d, the object recognition performance isimproved, and thus it is possible to notify a user of the collidingobject more rapidly and more reliably.

In addition, in step S208, if there is an object with which it isdetermined that “collision” will occur on the reference image of thescreen, the control unit generates a control signal for avoiding thecollision and outputs to outside of the imaging apparatus. Therefore,compared with a case of not satisfying the item d, the objectrecognition performance is improved, and therefore it is possible toperform control for avoiding the object more rapidly and more reliablyand thus to reduce a possibility of the collision.

-   -   Item 2-11: The transmittance of the optical element is high.    -   Item 2-12: The distortion coefficient of the lens in the radial        direction of the optical element is small.    -   Item 2-13: The distortion coefficient of the lens in the        tangential direction of the optical element is small.    -   Item 2-14: The dynamic range of the imaging element is wide.    -   Item 2-15: The luminance value of an image in uniform light of        the imaging element is great.    -   Item 2-16: The SN ratio of the imaging element is small.    -   Item 2-17: The standard deviation of luminance values of an        image in uniform light of the imaging element is small.    -   Item 2-18: The level of shot noise of light with predetermined        intensity of the imaging element is low.    -   Item 2-19: The level of dark current noise of the imaging        element is low.    -   Item 2-20: The level of reading noise of the imaging element is        low.    -   Item 2-21: The level of fixed pattern noise with predetermined        intensity of the imaging element is low.    -   Item 2-22: The number of defective pixels of the imaging element        is low.    -   Item 2-23: The dynamic range of the imaging element which        receives light having passed through the optical element is        wide.    -   Item 2-24: The luminance value of an image in uniform light of        the imaging element which receives light having passed through        the optical element is great.    -   Item 2-25: The SN ratio of the imaging element which receives        light having passed through the optical element is small.    -   Item 2-26: The standard deviation of luminance values of an        image in uniform light of the imaging element which receives        light having passed through the optical element is small.    -   Item 2-27: The level of shot noise of light with predetermined        intensity of the imaging element which receives light having        passed through the optical element is low.    -   Item 2-28: The level of dark current noise of the imaging        element which receives light having passed through the optical        element is low.    -   Item 2-29: The level of reading noise of the imaging element        which receives light having passed through the optical element        is low.    -   Item 2-30: The level of fixed pattern noise of light with        predetermined intensity of the imaging element which receives        light having passed through the optical element is low.

In operation procedures of the imaging apparatus according to theembodiment of the present invention illustrated in FIG. 4, thecharacteristic input/output means 423 reads, from the characteristicstorage means 404, the information regarding transmittance, anddistortion coefficients of the lenses in the radial direction and thetangential direction of the optical element 101 a and the opticalelement 101 b; the information regarding dynamic ranges, luminancevalues of images obtained by imaging an object with uniform light, SNratios, standard deviations (variations) of luminance values of imagesobtained by imaging an object with uniform light, shot noise of lightwith predetermined intensity, dark current noise, reading noise, fixedpattern noise of light with predetermined intensity, and the number ofdefective pixels of the imaging element 103 a and the imaging element103 b; and dynamic ranges of the imaging element 103 a and the imagingelement 103 b which receive light having passed through the opticalelement 101 a and the optical element 101 b, luminance values of imagesof the imaging element 103 a and the imaging element 103 b which receivelight having passed through the optical element 101 a and the opticalelement 101 b when an object is imaged with uniform light, SN ratios ofthe imaging element 103 a and the imaging element 103 b which receivelight having passed through the optical element 101 a and the opticalelement 101 b, standard deviations (variations) of luminance values ofimages of the imaging element 103 a and the imaging element 103 b whichreceive light having passed through the optical element 101 a and theoptical element 101 b when an object is imaged with uniform light, shotnoise of the imaging element 103 a and the imaging element 103 b whichreceive light having passed through the optical element 101 a and theoptical element 101 b when light with predetermined intensity isincident, dark current noise of the imaging element 103 a and theimaging element 103 b which receive light having passed through theoptical element 101 a and the optical element 101 b, reading noise ofthe imaging element 103 a and the imaging element 103 b which receivelight having passed through the optical element 101 a and the opticalelement 101 b, fixed pattern noise of the imaging element 103 a and theimaging element 103 b which receive light having passed through theoptical element 101 a and the optical element 101 b when light withpredetermined intensity is incident, and the like, and outputs theinformation to outside of the imaging apparatus. Therefore, on the basisof the information values, it is possible to check whether or not one ofthe above-described items 2-11 to 2-30 is satisfied.

Modification Example 2-2

In the imaging apparatus according to the embodiment of the presentinvention illustrated in FIG. 4, also in a case where the imagingelement 103 a and the imaging element 103 b are respectively storeinformation regarding sensitivity characteristics, noise, the number ofdefective pixels, and the like of the imaging element 103 a and theimaging element 103 b; and in the operation procedures of the imagingapparatus according to the embodiment of the present inventionillustrated in FIG. 4, the characteristic input/output means 423 readsthe information regarding sensitivity characteristics, noise, the numberof defective pixels, and the like of the imaging element 103 a and theimaging element 103 b, from the imaging element 103 a and the imagingelement 103 b, and may output the information to outside of the imagingapparatus, on the basis of the information values, it is possible tocheck whether or not one of the items 2-3 to 2-5 is satisfied.

In addition, in operation procedures of the imaging apparatus accordingto the embodiment of the present invention illustrated in FIG. 4, alsoin a case where the characteristic input/output means 423 reads, fromthe imaging element 103 a and the imaging element 103 b, the informationregarding dynamic ranges, luminance values of images obtained by imagingan object with uniform light, SN ratios, standard deviations(variations) of luminance values of images obtained by imaging an objectwith uniform light, shot noise of light with predetermined intensity,dark current noise, reading noise, fixed pattern noise of light withpredetermined intensity, and the number of defective pixels of theimaging element 103 a and the imaging element 103 b, and outputs theinformation to outside of the imaging apparatus, on the basis of theinformation values, it is possible to check whether or not one of theabove-described items 2-14 to 2-22 is satisfied.

In addition, in step 501 of the operation procedures (FIG. 5) of theembodiment of the imaging apparatus of the present invention illustratedin FIG. 4, also in a case where the reference image selection means 424sets the imaging unit having the imaging element satisfying one item camong the following items 2-3 to 2-5 which are predetermined conditionsset in advance as an imaging unit for a reference image on the basis ofthe information regarding sensitivity characteristics, noise, the numberof defective pixels, and the like of the imaging element 103 a and theimaging element 103 b, read from the imaging element 103 a and theimaging element 103 b, compared with a case of not satisfying the itemc, quality of a reference image becomes higher than that of a comparisonimage, and an object recognition process is performed in step 206 byusing the reference image with the higher quality, so that objectrecognition performance is improved. In addition, if an imagingapparatus is manufactured without taking the item c into consideration,a case of not satisfying the item c occurs. If an imaging apparatus ismanufactured so as to satisfy the item c, object recognition performanceis improved in this case compared with an imaging apparatus which ismanufactured without taking the item c into consideration, and thus itis possible to reduce variations in the object recognition performancefor each of imaging apparatuses.

In addition, in step 501 of the operation procedures (FIG. 5) of theimaging apparatus of the embodiment of the present invention illustratedin FIG. 4, also in a case where the reference image selection means 424sets the imaging unit having the imaging element satisfying one item damong the above-described items 2-14 to 2-22 set in advance as animaging unit for a reference image on the basis of the informationregarding dynamic ranges, luminance values of images obtained by imagingan object with uniform light, SN ratios, standard deviations(variations) of luminance values of images obtained by imaging an objectwith uniform light, shot noise of light with predetermined intensity,dark current noise, reading noise, fixed pattern noise of light withpredetermined intensity, and the number of defective pixels of theimaging element 103 a and the imaging element 103 b, read from theimaging element 103 a and the imaging element 103 b, compared with acase of not satisfying the item d, quality of a reference image becomeshigher than that of a comparison image, and an object recognitionprocess is performed in step 206 by using the reference image with thehigher quality, so that object recognition performance is improved. Inaddition, if an imaging apparatus is manufactured without taking theitem d into consideration, a case of not satisfying the item d occurs.If an imaging apparatus is manufactured so as to satisfy the item d,object recognition performance is improved in this case compared with animaging apparatus which is manufactured without taking the item d intoconsideration, and thus it is possible to reduce variations in theobject recognition performance for each of imaging apparatuses.

Modification Example 2-3

In the imaging apparatus according to the embodiment of the presentinvention illustrated in FIG. 4, in a case where the characteristicstorage means 104 a and the characteristic storage means 104 b arerespectively provided in the imaging unit 400 a and the imaging unit 400b instead of the characteristic storage means 404 of the calculationunit 410; and, in operation procedures of the imaging apparatusaccording to the embodiment of the present invention illustrated in FIG.4, the characteristic input/output means 423 reads, from thecharacteristic storage means 104 a and 104 b, the information regardingtransmittance, distortions, and the like of the optical element 101 aand the optical element 101 b, the information regarding sensitivitycharacteristics, noise, the number of defective pixels, and the like ofthe imaging element 103 a and the imaging element 103 b, and theinformation regarding sensitivity characteristics, noise, and the likeof the imaging unit 400 a and the imaging unit 400 b, stored in thecharacteristic storage means 104 a and the characteristic storage means104 b, and outputs the information to outside of the imaging apparatus,on the basis of the information values, it is possible to check whetheror not one of the above-described items 2-1 to 2-7 is satisfied.

In addition, in operation procedures of the imaging apparatus accordingto the embodiment of the present invention illustrated in FIG. 4, alsoin a case where the characteristic input/output means 423 reads, fromthe characteristic storage means 104 a and 104 b, the informationregarding transmittance, and distortion coefficients of the lenses inthe radial direction and the tangential direction of the optical element101 a and the optical element 101 b; the information regarding dynamicranges, luminance values of images obtained by imaging an object withuniform light, SN ratios, standard deviations (variations) of luminancevalues of images obtained by imaging an object with uniform light, shotnoise of light with predetermined intensity, dark current noise, readingnoise, fixed pattern noise of light with predetermined intensity, andthe number of defective pixels of the imaging element 103 a and theimaging element 103 b; and dynamic ranges of the imaging element 103 aand the imaging element 103 b which receive light having passed throughthe optical element 101 a and the optical element 101 b, luminancevalues of images of the imaging element 103 a and the imaging element103 b which receive light having passed through the optical element 101a and the optical element 101 b when an object is imaged with uniformlight, SN ratios of the imaging element 103 a and the imaging element103 b which receive light having passed through the optical element 101a and the optical element 101 b, standard deviations (variations) ofluminance values of images of the imaging element 103 a and the imagingelement 103 b which receive light having passed through the opticalelement 101 a and the optical element 101 b when an object is imagedwith uniform light, shot noise of the imaging element 103 a and theimaging element 103 b which receive light having passed through theoptical element 101 a and the optical element 101 b when light withpredetermined intensity is incident, dark current noise of the imagingelement 103 a and the imaging element 103 b which receive light havingpassed through the optical element 101 a and the optical element 101 b,reading noise of the imaging element 103 a and the imaging element 103 bwhich receive light having passed through the optical element 101 a andthe optical element 101 b, fixed pattern noise of the imaging element103 a and the imaging element 103 b which receive light having passedthrough the optical element 101 a and the optical element 101 b whenlight with predetermined intensity is incident, and the like, andoutputs the information to outside of the imaging apparatus, on thebasis of the information values, it is possible to check whether or notone of the above-described items 2-1 to 2-30 is satisfied.

In addition, in step 501 of the operation procedures (FIG. 5) of theimaging apparatus of the embodiment of the present invention illustratedin FIG. 4, also in a case where the reference image selection means 424sets the imaging unit having the imaging element satisfying one item camong the following items 2-1 to 2-7 set in advance as an imaging unitfor a reference image on the basis of the information regardingtransmittance, distortions, and the like of the optical element 101 aand the optical element 101 b, the information regarding sensitivitycharacteristics, noise, the number of defective pixels, and the like ofthe imaging element 103 a and the imaging element 103 b, and theinformation regarding sensitivity characteristics, noise, and the likeof the imaging unit 400 a and the imaging unit 400 b, read from thecharacteristic storage means 104 a and the characteristic storage means104 b, compared with a case of not satisfying the item c, quality of areference image becomes higher than that of a comparison image, and anobject recognition process is performed in step 206 by using thereference image with the higher quality, so that object recognitionperformance is improved. In addition, if an imaging apparatus ismanufactured without taking the item c into consideration, a case of notsatisfying the item c occurs. If an imaging apparatus is manufactured soas to satisfy the item c, object recognition performance is improved inthis case compared with an imaging apparatus which is manufacturedwithout taking the item c into consideration, and thus it is possible toreduce variations in the object recognition performance for each ofimaging apparatuses.

In addition, in step 501 of the operation procedures (FIG. 5) of theimaging apparatus of the embodiment of the present invention illustratedin FIG. 4, also in a case where the reference image selection means 424sets the imaging unit having the imaging element satisfying one item damong the above-described items 2-11 to 2-30 set in advance as animaging unit for a reference image on the basis of the informationregarding transmittance, and distortion coefficients of the lenses inthe radial direction and the tangential direction of the optical element101 a and the optical element 101 b; the information regarding dynamicranges, luminance values of images obtained by imaging an object withuniform light, SN ratios, standard deviations (variations) of luminancevalues of images obtained by imaging an object with uniform light, shotnoise of light with predetermined intensity, dark current noise, readingnoise, fixed pattern noise of light with predetermined intensity, andthe number of defective pixels of the imaging element 103 a and theimaging element 103 b; and dynamic ranges of the imaging element 103 aand the imaging element 103 b which receive light having passed throughthe optical element 101 a and the optical element 101 b, luminancevalues of images of the imaging element 103 a and the imaging element103 b which receive light having passed through the optical element 101a and the optical element 101 b when an object is imaged with uniformlight, SN ratios of the imaging element 103 a and the imaging element103 b which receive light having passed through the optical element 101a and the optical element 101 b, standard deviations (variations) ofluminance values of images of the imaging element 103 a and the imagingelement 103 b which receive light having passed through the opticalelement 101 a and the optical element 101 b when an object is imagedwith uniform light, shot noise of the imaging element 103 a and theimaging element 103 b which receive light having passed through theoptical element 101 a and the optical element 101 b when light withpredetermined intensity is incident, dark current noise of the imagingelement 103 a and the imaging element 103 b which receive light havingpassed through the optical element 101 a and the optical element 101 b,reading noise of the imaging element 103 a and the imaging element 103 bwhich receive light having passed through the optical element 101 a andthe optical element 101 b, fixed pattern noise of the imaging element103 a and the imaging element 103 b which receive light having passedthrough the optical element 101 a and the optical element 101 b whenlight with predetermined intensity is incident, and the like, read fromthe imaging element 103 a and the imaging element 103 b, compared with acase of not satisfying the item d, quality of a reference image becomeshigher than that of a comparison image, and an object recognitionprocess is performed in step 206 by using the reference image with thehigher quality, so that object recognition performance is improved. Inaddition, if an imaging apparatus is manufactured without taking theitem d into consideration, a case of not satisfying the item d occurs.If an imaging apparatus is manufactured so as to satisfy the item d,object recognition performance is improved in this case compared with animaging apparatus which is manufactured without taking the item d intoconsideration, and thus it is possible to reduce variations in theobject recognition performance for each of imaging apparatuses.

Modification Example 2-4

In step 501 of the imaging apparatus according to the embodiment of thepresent invention illustrated in FIG. 4, the reference image selectionmeans 424 sends a signal indicating that “an image is acquired when theshutter means is closed”, to the image acquisition means 417 a and theimage acquisition means 417 b. Right after receiving the synchronizationsignal from the synchronization signal transmission means 116, the imageacquisition means 417 a receives an image of light refracted by theoptical element 101 a in a state in which the shutter mechanism of theshutter means 102 a is closed, so as to generate an image correspondingto the intensity of the light, and sends the image to the imageacquisition means 417 a. The image acquisition means 417 a receives theimage from the imaging element 103 a and sends the image to thereference image selection means 424. Right after receiving thesynchronization signal from the synchronization signal transmissionmeans 116, the image acquisition means 417 b receives an image of lightrefracted by the optical element 101 b in a state in which the shuttermechanism of the shutter means 102 b is closed, so as to generate animage corresponding to the intensity of the light, and sends the imageto the image acquisition means 417 b. The image acquisition means 417 breceives the image from the imaging element 103 b and sends the image tothe reference image selection means 424. The reference image selectionmeans 424 receives the images from the image acquisition means 417 a andthe image acquisition means 417 b, respectively. In a case where aluminance value of each pixel of each image is equal to or greater thana threshold value, it is determined that the pixel is a defective pixel,and the number of defective pixels is detected for each image. In a casewhere an imaging unit generating an image having the smaller number ofdefective pixels is used as the imaging unit for a reference image, evenif the number of defective pixels increases due to deterioration overtime in the imaging element, the imaging unit for a reference image isdetermined on the basis of a real image, and thus an image having asmall number of defective pixels can be correctly set as a referenceimage. Therefore, it is possible to prevent deterioration in recognitionperformance using a reference image.

In addition, if an imaging apparatus is manufactured without taking intoconsideration the item d which leads to a small number of defectivepixels, a case of not satisfying the item d occurs. If an imagingapparatus is manufactured so as to satisfy the item d, objectrecognition performance is improved in this case compared with animaging apparatus which is manufactured without taking the item d intoconsideration, and thus it is possible to reduce variations in theobject recognition performance for each of imaging apparatuses. Inaddition, in step 207, the screen/sound output unit 130 displays a framewith a predetermined color in an object with which it is determined that“collision” will occur on the reference image of the screen, and outputsa warning sound. Therefore, compared with a case of not satisfying theitem d, the object recognition performance is improved, and thus it ispossible to notify a user of the colliding object more rapidly and morereliably. In addition, if there is an object with which it is determinedthat “collision” will occur on the reference image of the screen in step208, the control unit generates a control signal for avoiding thecollision and outputs to outside of the imaging apparatus. Therefore,compared with a case of not satisfying the item d, the objectrecognition performance is improved, and therefore it is possible toperform control for avoiding the object more rapidly and more reliablyand thus to reduce a possibility of the collision.

Modification Example 2-5

In the operation procedures of the imaging apparatus according to theembodiment of the present invention illustrated in FIG. 4, steps 601 and602 are added thereto as illustrated in FIG. 6. Hereinafter, steps 601and 602 will be described.

Step 601: If an imaging unit for a reference image is re-determined instep 602, the flow proceeds to step 503. If an imaging unit for areference image is not re-determined, the flow proceeds to step 602.

Step 602: The reference image selection means 424 receives the imagesfrom the image acquisition means 417 a and the image acquisition means417 b. Each of the images is divided into a plurality of regions. A meanof luminance values of each region of each image is calculated, and if aregion having the greatest luminance mean and a region having thesmallest luminance mean of each image are the same in the two images,the following determination is performed. An image in which a luminancemean difference between the region having the greatest luminance meanand the region having the smallest luminance mean in each image islarger is determined as being an image in which a dynamic range of theimaging element which receives light having passed through the opticalelement is wide. The imaging unit related to the image determined asbeing an image in which a dynamic range of the imaging element whichreceives light having passed through the optical element is wide is setas an imaging unit for a reference image.

As illustrated in step 602, in a case where the imaging unit for areference image is set, even if a dynamic range of the imaging elementwhich receives light having passed through the optical element changesdue to deterioration over time in the imaging element, the imaging unitfor a reference image is determined on the basis of a real image, andthus an image in which a dynamic range of the imaging element whichreceives light having passed through the optical element is wide can becorrectly set as a reference image. Therefore, it is possible to preventdeterioration in recognition performance using a reference image.

In addition, if an imaging apparatus is manufactured without taking intoconsideration the item d which leads to a wide dynamic range of animaging element which receives light having passed through an opticalelement, a case of not satisfying the item d occurs. If an imagingapparatus is manufactured so as to satisfy the item d, objectrecognition performance is improved in this case compared with animaging apparatus which is manufactured without taking the item d intoconsideration, and thus it is possible to reduce variations in theobject recognition performance for each of imaging apparatuses.

In addition, in step 207, the screen/sound output unit 130 displays aframe with a predetermined color in an object with which it isdetermined that “collision” will occur on the reference image of thescreen, and outputs a warning sound. Therefore, compared with a case ofnot satisfying the item d, the object recognition performance isimproved, and thus it is possible to notify a user of the collidingobject more rapidly and more reliably.

In addition, if there is an object with which it is determined that“collision” will occur on the reference image of the screen in step 208,the control unit generates a control signal for avoiding the collisionand outputs to outside of the imaging apparatus. Therefore, comparedwith a case of not satisfying the item d, the object recognitionperformance is improved, and therefore it is possible to perform controlfor avoiding the object more rapidly and more reliably and thus toreduce a possibility of the collision.

Modification Example 2-6

In the operation procedures of the imaging apparatus according to theembodiment of the present invention illustrated in FIG. 4, steps 601 and602 are added thereto as illustrated in FIG. 6. Hereinafter, steps 601and 602 will be described.

Step 601: If an imaging unit for a reference image is re-determined instep 602, the flow proceeds to step 503. If an imaging unit for areference image is not re-determined, the flow proceeds to step 602.

Step 602: The reference image selection means 424 receives the imagesfrom the image acquisition means 417 a and the image acquisition means417 b. Each of the images is divided into a plurality of regions. Ifregions having the highest luminance in the respective images are thesame in the two images, a mean and a standard deviation of luminancevalues of the region in each image are calculated, and if these valuesare all within threshold values, it is determined that uniform light isincident to the region, and the following process is performed. An imagein which the mean of luminance values of the region is greater isdetermined as being an image in which a luminance value an image inuniform light of the imaging element which receives light having passedthrough the optical element is great. The imaging unit related to theimage determined as being an image in which a luminance value an imagein uniform light of the imaging element which receives light havingpassed through the optical element is great is set as an imaging unitfor a reference image.

As illustrated in step 602, in a case where the imaging unit for areference image is set, even if a luminance value of an image in uniformlight of the imaging element which receives light having passed throughthe optical element changes due to deterioration over time in theimaging element, the imaging unit for a reference image is determined onthe basis of a real image, and thus an image in which a luminance valuean image in uniform light of the imaging element which receives lighthaving passed through the optical element is great can be correctly setas a reference image. Therefore, it is possible to prevent deteriorationin recognition performance using a reference image.

In addition, if an imaging apparatus is manufactured without taking intoconsideration the item d which leads to a great luminance value of animage in uniform light of an imaging element which receives light havingpassed through an optical element, a case of not satisfying the item doccurs. If an imaging apparatus is manufactured so as to satisfy theitem d, object recognition performance is improved in this case comparedwith an imaging apparatus which is manufactured without taking the itemd into consideration, and thus it is possible to reduce variations inthe object recognition performance for each of imaging apparatuses.

Further, in step 207, the screen/sound output unit 130 displays a framewith a predetermined color in an object with which it is determined that“collision” will occur on the reference image of the screen, and outputsa warning sound. Therefore, compared with a case of not satisfying theitem d, the object recognition performance is improved, and thus it ispossible to notify a user of the colliding object more rapidly and morereliably.

In addition, if there is an object with which it is determined that“collision” will occur on the reference image of the screen in step 208,the control unit generates a control signal for avoiding the collisionand outputs to outside of the imaging apparatus. Therefore, comparedwith a case of not satisfying the item d, the object recognitionperformance is improved, and therefore it is possible to perform controlfor avoiding the object more rapidly and more reliably and thus toreduce a possibility of the collision.

Modification Example 2-7

In the operation procedures of the imaging apparatus according to theembodiment of the present invention illustrated in FIG. 4, steps 601 and602 are added thereto as illustrated in FIG. 6. Hereinafter, steps 601and 602 will be described.

Step 601: If an imaging unit for a reference image is re-determined instep 602, the flow proceeds to step 503. If an imaging unit for areference image is not re-determined, the flow proceeds to step 602.

Step 602: The reference image selection means 424 receives the imagesfrom the image acquisition means 417 a and the image acquisition means417 b. Each of the images is divided into a plurality of regions. Ifregions having the highest luminance in the respective images are thesame in the two images, a mean and a standard deviation of luminancevalues of the region in each image are calculated, and if these valuesare all within threshold values, it is determined that uniform light isincident to the region, and the following determination is performed. Animage in which the standard deviation of luminance values of the regionis smaller is determined as being an image in which a standard deviationof luminance values of an image in uniform light of the imaging elementwhich receives light having passed through the optical element is small.The imaging unit related to an image in which a standard deviation ofluminance values of an image in uniform light of the imaging elementwhich receives light having passed through the optical element is smallis set as an imaging unit for a reference image.

As illustrated in step 602, in a case where the imaging unit for areference image is set, even if a standard deviation of luminance valuesof an image in uniform light of the imaging element which receives lighthaving passed through the optical element changes due to deteriorationover time in the imaging element, the imaging unit for a reference imageis determined on the basis of a real image, and thus an image in which astandard deviation of luminance values of an image in uniform light ofthe imaging element which receives light having passed through theoptical element is small can be correctly set as a reference image.Therefore, it is possible to prevent deterioration in recognitionperformance using a reference image.

In addition, if an imaging apparatus is manufactured without taking intoconsideration the item d which leads to a small standard deviation ofluminance values of an image in uniform light of an imaging elementwhich receives light having passed through an optical element, a case ofnot satisfying the item d occurs. If an imaging apparatus ismanufactured so as to satisfy the item d, object recognition performanceis improved in this case compared with an imaging apparatus which ismanufactured without taking the item d into consideration, and thus itis possible to reduce variations in the object recognition performancefor each of imaging apparatuses.

In addition, in step 207, the screen/sound output unit 130 displays aframe with a predetermined color in an object with which it isdetermined that “collision” will occur on the reference image of thescreen, and outputs a warning sound. Therefore, compared with a case ofnot satisfying the item d, the object recognition performance isimproved, and thus it is possible to notify a user of the collidingobject more rapidly and more reliably.

In addition, if there is an object with which it is determined that“collision” will occur on the reference image of the screen in step 208,the control unit generates a control signal for avoiding the collisionand outputs to outside of the imaging apparatus. Therefore, comparedwith a case of not satisfying the item d, the object recognitionperformance is improved, and therefore it is possible to perform controlfor avoiding the object more rapidly and more reliably and thus toreduce a possibility of the collision.

Modification Example 2-7

In the imaging apparatus according to the embodiment of the presentinvention illustrated in FIG. 1, not only two imaging units but alsothree or more imaging units are provided, and the operation proceduresillustrated in FIG. 5 or 6 are performed on a combination of two of theplurality of imaging units.

Therefore, a distance to an object imaged by the plurality of imagingunits or the object can be recognized, and, in the combination of thetwo of the plurality of imaging units, in step 501, the reference imageselection means 424 sets the imaging unit having the optical element orthe imaging element satisfying one item c of the above-described presetitems 2-1 to 2-7 as an imaging unit for a reference image, and, in step503, the reference image selection means 424 sets the image of theimaging unit for a reference image, set in step 501, as a referenceimage and sets the other image as a comparison image. Therefore,compared with a case of not satisfying the item c, quality of areference image becomes higher than that of a comparison image, and anobject recognition process is performed in step 206 by using thereference image with the higher quality, so that object recognitionperformance is improved.

In addition, if an imaging apparatus is manufactured without taking theitem c into consideration, a case of not satisfying the item c occurs.If an imaging apparatus is manufactured so as to satisfy the item c,object recognition performance is improved in this case compared with animaging apparatus which is manufactured without taking the item c intoconsideration, and thus it is possible to reduce variations in theobject recognition performance for each of imaging apparatuses.

In addition, in step 207, the screen/sound output unit 130 displays aframe with a predetermined color in an object with which it isdetermined that “collision” will occur on the reference image of thescreen, and outputs a warning sound. Therefore, compared with a case ofnot satisfying the item c, the object recognition performance isimproved, and thus it is possible to notify a user of the collidingobject more rapidly and more reliably.

In addition, in step S208, if there is an object with which it isdetermined that “collision” will occur on the reference image of thescreen, the control unit generates a control signal for avoiding thecollision and outputs to outside of the imaging apparatus. Therefore,compared with a case of not satisfying the item c, the objectrecognition performance is improved, and therefore it is possible toperform control for avoiding the object more rapidly and more reliablyand thus to reduce a possibility of the collision.

REFERENCE SIGNS LIST

-   -   100 a IMAGING UNIT    -   100 b IMAGING UNIT    -   101 a OPTICAL ELEMENT    -   101 b OPTICAL ELEMENT    -   102 a SHUTTER MEANS    -   102 b SHUTTER MEANS    -   103 a IMAGING ELEMENT    -   103 b IMAGING ELEMENT    -   104 a CHARACTERISTIC STORAGE MEANS    -   104 b CHARACTERISTIC STORAGE. MEANS    -   110 CALCULATION UNIT    -   111 REFERENCE IMAGE STORAGE MEANS    -   112 COMPARISON IMAGE STORAGE MEANS    -   113 PROCESSED IMAGE STORAGE MEANS    -   114 LUMINANCE CORRECTION INFORMATION STORAGE MEANS    -   115 GEOMETRIC RECTIFICATION INFORMATION STORAGE MEANS    -   116 SYNCHRONIZATION SIGNAL TRANSMISSION MEANS    -   117 a REFERENCE IMAGE ACQUISITION MEANS    -   117 b COMPARISON IMAGE ACQUISITION MEANS    -   118 LUMINANCE CORRECTION MEANS    -   119 GEOMETRIC RECTIFICATION MEANS    -   120 DISPARITY CALCULATION MEANS    -   121 DISTANCE CALCULATION MEANS    -   122 RECOGNITION MEANS    -   130 SCREEN/SOUND OUTPUT UNIT    -   140 CONTROL UNIT    -   301 REFERENCE IMAGE    -   302 COMPARISON IMAGE    -   400 a IMAGING UNIT    -   400 b IMAGING UNIT    -   404 CHARACTERISTIC STORAGE MEANS

1. An imaging apparatus comprising: a first optical element; a firstimaging element that receives light having passed through the firstoptical element, and outputs an image which has a luminance valuecorresponding to intensity of the light and is processed as a referenceimage; a second optical element; a second imaging element that receiveslight having passed through the second optical element, and outputs animage which has a luminance value corresponding to intensity of thelight and is processed as a comparison image; distance calculation meansfor calculating a distance image on the basis of the reference image andthe comparison image; and recognition means for recognizing an object onthe basis of the distance image calculated by the distance calculationmeans, wherein the first optical element and the second optical element,or the first imaging element and the second imaging element satisfy atleast one of conditions in which transmittance of the first opticalelement is higher than transmittance of the second optical element; adistortion of the first optical element is smaller than a distortion ofthe second optical element; a sensitivity characteristic of the firstimaging element is higher than a sensitivity characteristic of thesecond imaging element; a level of noise of the first imaging element islower than a level of noise of the second imaging element; the number ofdefective pixels of the first imaging element is smaller than the numberof defective pixels of the second imaging element; a sensitivitycharacteristic of the first imaging element which receives light havingpassed through the first optical element is higher than a sensitivitycharacteristic of the second imaging element which receives light havingpassed through the second optical element; and a level of noise of thefirst imaging element which receives light having passed through thefirst optical element is lower than a level of noise of the secondimaging element which receives light having passed through the secondoptical element.
 2. The imaging apparatus according to claim 1, whereinthe first optical element and the second optical element, or the firstimaging element and the second imaging element satisfy at least one ofconditions in which transmittance of the first optical element is higherthan transmittance of the second optical element; a distortioncoefficient of a lens in a radial direction of the first optical elementis smaller than distortion coefficient of a lens in a radial directionof the second optical element; a distortion coefficient of a lens in atangential direction of the first optical element is smaller thandistortion coefficient of a lens in a tangential direction of the secondoptical element; a dynamic range of the first imaging element is widerthan a dynamic range of the second imaging element; a luminance value ofan image in uniform light of the first imaging element is greater than aluminance value of an image in uniform light of the second imagingelement; an SN ratio of the first imaging element is smaller than an SNratio of the second imaging element; a standard deviation of luminancevalues of an image in uniform light of the first imaging element issmaller than a standard deviation of luminance values of an image inuniform light of the second imaging element; the number of defectivepixels of the first imaging element is smaller than the number ofdefective pixels of the second imaging element; a dynamic range of thefirst imaging element which receives light having passed through thefirst optical element is wider than a dynamic range of the secondimaging element which receives light having passed through the secondoptical element; a luminance value of an image in uniform light of thefirst imaging element which receives light having passed through thefirst optical element is greater than a luminance value of an image inuniform light of the second imaging element which receives light havingpassed through the second optical element; an SN ratio of the firstimaging element which receives light having passed through the firstoptical element is smaller than an SN ratio of the second imagingelement which receives light having passed through the second opticalelement; and a standard deviation of luminance values of an image inuniform light of the first imaging element which receives light havingpassed through the first optical element is smaller than a standarddeviation of luminance values of an image in uniform light of the secondimaging element which receives light having passed through the secondoptical element.
 3. The imaging apparatus according to claim 1, furthercomprising: characteristic storage means for storing at least one of thetransmittance or the distortions of the first optical element and thesecond optical element, the sensitivity characteristics, the levels ofnoise, or the number of defective pixels of the first imaging elementand the second imaging element, and the sensitivity characteristics orthe levels of noise of the first imaging element which receives lighthaving passed through the first optical element and the second imagingelement which receives light having passed through the second opticalelement.
 4. The imaging apparatus according to claim 3, wherein thecharacteristic storage means stores at least one of the transmittance ofthe first optical element and the second optical element; the distortioncoefficients of the lenses in the radial direction of the first opticalelement and the second optical element; the distortion coefficients ofthe lenses in the tangential direction of the first optical element andthe second optical element; the dynamic ranges of the first imagingelement and the second imaging element; the luminance values of imagesin uniform light of the first imaging element and the second imagingelement; the SN ratios of the first imaging element and the secondimaging element; the standard deviations of luminance values of imagesin uniform light of the first imaging element and the second imagingelement; the number of defective pixels of the first imaging element andthe second imaging element; the dynamic ranges of the first imagingelement which receives light having passed through the first opticalelement and the second imaging element which receives light havingpassed through the second optical element; the luminance values ofimages in uniform light of the first imaging element which receiveslight having passed through the first optical element and the secondimaging element which receives light having passed through the secondoptical element; the SN ratios of the first imaging element whichreceives light having passed through the first optical element and thesecond imaging element which receives light having passed through thesecond optical element; and the standard deviations of luminance valuesof images in uniform light of the first imaging element which receiveslight having passed through the first optical element and the secondimaging element which receives light having passed through the secondoptical element.
 5. The imaging apparatus according to claim 1, whereinthe first imaging element stores at least one of the sensitivitycharacteristic of the first imaging element, the level of noise of thefirst imaging element, and the number of defective pixels of the firstimaging element, and wherein the second imaging element stores at leastone of the sensitivity characteristic of the second imaging element, thelevel of noise of the second imaging element, and the number ofdefective pixels of the second imaging element.
 6. The imaging apparatusaccording to claim 1, wherein the first imaging element stores at leastone of a dynamic range of the first imaging element, a luminance valueof an image in uniform light of the first imaging element, an SN ratioof the first imaging element, a standard deviation of luminance valuesof an image in uniform light of the first imaging element, and thenumber of defective pixels of the first imaging element, and wherein thesecond imaging element stores at least one of a dynamic range of thesecond imaging element, a luminance value of an image in uniform lightof the second imaging element, an SN ratio of the second imagingelement, a standard deviation of luminance values of an image in uniformlight of the second imaging element, and the number of defective pixelsof the second imaging element.
 7. An imaging apparatus comprising: afirst optical element; a first imaging element that receives lighthaving passed through the first optical element, and outputs a firstimage which has a luminance value corresponding to intensity of thelight; a second optical element; a second imaging element that receiveslight having passed through the second optical element, and outputs asecond image which has a luminance value corresponding to intensity ofthe light; reference image selection means for selecting one imagesatisfying a predetermined condition, of the first image and the secondimage, as a reference image, and selecting the other image as acomparison image; distance calculation means for calculating a distanceimage on the basis of the reference image and the comparison image; andrecognition means for recognizing an object on the basis of the distanceimage calculated by the distance calculation means, wherein thepredetermined condition in the reference image selection means isrelated to one of an image in which transmittance is higher when thetransmittance of the first optical element is compared with thetransmittance of the second optical element; an image in which adistortion is smaller when the distortion of the first optical elementis compared with the distortion of the second optical element; an imagein which a sensitivity characteristic is higher when the sensitivitycharacteristic of the first imaging element is compared with thesensitivity characteristic of the second imaging element; an image inwhich a level of noise is lower when the level of noise of the firstimaging element is compared with the level of noise of the secondimaging element; an image in which the number of defective pixels issmaller when the number of defective pixels of the first imaging elementis compared with the number of defective pixels of the second imagingelement; an image in which a sensitivity characteristic is higher whenthe sensitivity characteristic of the first imaging element whichreceives light having passed through the first optical element iscompared with the sensitivity characteristic of the second imagingelement which receives light having passed through the second opticalelement; and an image in which a level of noise is lower when the levelof noise of the first imaging element which receives light having passedthrough the first optical element is compared with the level of noise ofthe second imaging element which receives light having passed throughthe second optical element.
 8. An imaging apparatus comprising: a firstoptical element; a first imaging element that receives light havingpassed through the first optical element, and outputs a first imagewhich has a luminance value corresponding to intensity of the light; asecond optical element; a second imaging element that receives lighthaving passed through the second optical element, and outputs a secondimage which has a luminance value corresponding to intensity of thelight; characteristic storage means for storing at least one piece ofcharacteristic information such as distortions of the first opticalelement and the second optical element, sensitivity characteristics,levels of noise, and the number of defective pixels of the first imagingelement and the second imaging element, and sensitivity characteristicsand the levels of noise of the first imaging element which receiveslight having passed through the first optical element and the secondimaging element which receives light having passed through the secondoptical element; reference image selection means for selecting one imagesatisfying a predetermined condition, as a reference image, andselecting the other image as a comparison image, on the basis of thecharacteristic information stored in the characteristic storage means;distance calculation means for calculating a distance image on the basisof the reference image and the comparison image; and recognition meansfor recognizing an object on the basis of the distance image calculatedby the distance calculation means, wherein the predetermined conditionin the reference image selection means is related to one of an image inwhich transmittance is higher when the transmittance of the firstoptical element is compared with the transmittance of the second opticalelement; an image in which a distortion is smaller when the distortionof the first optical element is compared with the distortion of thesecond optical element; an image in which a sensitivity characteristicis higher when the sensitivity characteristic of the first imagingelement is compared with the sensitivity characteristic of the secondimaging element; an image in which a level of noise is lower when thelevel of noise of the first imaging element is compared with the levelof noise of the second imaging element; an image in which the number ofdefective pixels is smaller when the number of defective pixels of thefirst imaging element is compared with the number of defective pixels ofthe second imaging element; an image in which a sensitivitycharacteristic is higher when the sensitivity characteristic of thefirst imaging element which receives light having passed through thefirst optical element is compared with the sensitivity characteristic ofthe second imaging element which receives light having passed throughthe second optical element; and an image in which a level of noise islower when the level of noise of the first imaging element whichreceives light having passed through the first optical element iscompared with the level of noise of the second imaging element whichreceives light having passed through the second optical element.
 9. Theimaging apparatus according to claim 7, wherein the predeterminedcondition in the reference image selection means is related to one of animage in which transmittance is higher when the transmittance of thefirst optical element is compared with the transmittance of the secondoptical element; an image in which a distortion coefficient is smallerwhen the distortion coefficient of a lens in a radial direction of thefirst optical element is compared with the distortion coefficient of alens in a radial direction of the second optical element; an image inwhich a distortion coefficient is smaller when the distortioncoefficient of a lens in a tangential direction of the first opticalelement is compared with the distortion coefficient of a lens in atangential direction of the second optical element; an image in which adynamic range is wider when the dynamic range of the first imagingelement is compared with the dynamic range of the second imagingelement; an image in which a luminance value is greater when theluminance value of an image in uniform light of the first imagingelement is compared with the luminance value of an image in uniformlight of the second imaging element; an image in which an SN ratio isgreater when the SN ratio of the first imaging element is compared withthe SN ratio of the second imaging element; an image in which a standarddeviation is smaller when the standard deviation of luminance values ofan image in uniform light of the first imaging element is compared withthe standard deviation of luminance values of an image in uniform lightof the second imaging element; an image in which the number of defectivepixels is smaller when the number of defective pixels of the firstimaging element is compared with the number of defective pixels of thesecond imaging element; an image in which a dynamic range is wider whenthe dynamic range of the first imaging element which receives lighthaving passed through the first optical element is compared with thedynamic range of the second imaging element which receives light havingpassed through the second optical element; an image in which a luminancevalue is greater when the luminance value of an image in uniform lightof the first imaging element which receives light having passed throughthe first optical element is compared with the luminance value of animage in uniform light of the second imaging element which receiveslight having passed through the second optical element; an image inwhich an SN ratio is smaller when the SN ratio of the first imagingelement which receives light having passed through the first opticalelement is compared with the SN ratio of the second imaging elementwhich receives light having passed through the second optical element;and an image in which a standard deviation is smaller when the standarddeviation of luminance values of an image in uniform light of the firstimaging element which receives light having passed through the firstoptical element is compared with the standard deviation of luminancevalues of an image in uniform light of the second imaging element whichreceives light having passed through the second optical element.
 10. Theimaging apparatus according to claim 8, wherein the characteristicstorage means stores at least one piece of characteristic informationsuch as transmittance of the first optical element and the secondoptical element; distortion coefficients of the lenses in the radialdirection of the first optical element and the second optical element;distortion coefficients of the lenses in the tangential direction of thefirst optical element and the second optical element; dynamic ranges ofthe first imaging element and the second imaging element; luminancevalues of images in uniform light of the first imaging element and thesecond imaging element; SN ratios of the first imaging element and thesecond imaging element; standard deviations of luminance values ofimages in uniform light of the first imaging element and the secondimaging element; the number of defective pixels of the first imagingelement and the second imaging element; dynamic ranges of the firstimaging element which receives light having passed through the firstoptical element and the second imaging element which receives lighthaving passed through the second optical element; luminance values ofimages in uniform light of the first imaging element which receiveslight having passed through the first optical element and the secondimaging element which receives light having passed through the secondoptical element; SN ratios of the first imaging element which receiveslight having passed through the first optical element and the secondimaging element which receives light having passed through the secondoptical element; and standard deviations of luminance values of imagesin uniform light of the first imaging element which receives lighthaving passed through the first optical element and the second imagingelement which receives light having passed through the second opticalelement.
 11. The imaging apparatus according to claim 7, furthercomprising: first shutter means for causing light having passed throughthe first optical element to pass therethrough only for a predeterminedexposure time; and second shutter means for causing light having passedthrough the second optical element to pass therethrough only for apredetermined exposure time, wherein, in the first image and the secondimage when the first shutter means and the second shutter means areclosed, the reference image selection means determines a pixel having aluminance value which is equal to or greater than a threshold value as adefective pixel, calculates the number of defective pixels of each ofthe first image and the second image, selects an image in which thenumber of defective pixels is smaller as a reference image, and selectsthe other image as a comparison image.
 12. The imaging apparatusaccording to claim 7, wherein the reference image selection meansdivides each of the first image and the second image into a plurality ofregions; calculates a mean of luminance values of each region;determines an image in which a luminance value mean difference betweenthe region of which the luminance value mean is greatest and the regionof which the luminance value mean is smallest in each image is larger asbeing an image in which a dynamic range of the imaging element whichreceives light having passed through the optical element is wide if aregion of which the calculated mean of the luminance values is greatestand a region of which the calculated means of the luminance values issmallest are the same in the first image and the second image; selectsthe image determined as being an image in which a dynamic range of theimaging element which receives light having passed through the opticalmeans is wide, as a reference image; and selects the other image as acomparison image.
 13. The imaging apparatus according to claim 7,wherein the reference image selection means divides each of the firstimage and the second image into a plurality of regions; calculates amean of luminance values of each region; calculates a mean and astandard deviation of the luminance values of the region having thehighest luminance if a region of which the calculated mean of theluminance values is greatest and a region of which the calculated meansof the luminance values is smallest are the same in the first image andthe second image; determines that uniform light is incident to theregion having the highest luminance, and determines an image in whichthe mean of the luminance values of the region having the highestluminance is greater as being an image in which a luminance value of animage in uniform light of the imaging element which receives lighthaving passed through the optical element is great, if both the mean andthe standard deviation of the luminance values of the region having thehighest luminance are within a predefined threshold value; selects theimage determined as being an image in which a luminance value of animage in uniform light of the imaging means which receives light havingpassed through the optical element is great, as a reference image; andselects the other image as a comparison image.
 14. The imaging apparatusaccording to claim 7, wherein the reference image selection meansdivides each of the first image and the second image into a plurality ofregions; calculates a mean of luminance values of each region;calculates a mean and a standard deviation of the luminance values ofthe region having the highest luminance if a region of which thecalculated mean of the luminance values is greatest and a region ofwhich the calculated means of the luminance values is smallest are thesame in the first image and the second image; determines that uniformlight is incident to the region having the highest luminance, anddetermines an image in which the standard deviation of the luminancevalues of the region having the highest luminance is smaller as being animage in which a standard deviation of an image in uniform light of theimaging element which receives light having passed through the opticalelement is small, if both the mean and the standard deviation of theluminance values of the region having the highest luminance are within apredefined threshold value; selects the image determined as being animage in which a luminance value of an image in uniform light of theimaging element which receives light having passed through the opticalelement is great, as a reference image; and selects the other image as acomparison image.
 15. The imaging apparatus according to claim 8,wherein the predetermined condition in the reference image selectionmeans is related to one of an image in which transmittance is higherwhen the transmittance of the first optical element is compared with thetransmittance of the second optical element; an image in which adistortion coefficient is smaller when the distortion coefficient of alens in a radial direction of the first optical element is compared withthe distortion coefficient of a lens in a radial direction of the secondoptical element; an image in which a distortion coefficient is smallerwhen the distortion coefficient of a lens in a tangential direction ofthe first optical element is compared with the distortion coefficient ofa lens in a tangential direction of the second optical element; an imagein which a dynamic range is wider when the dynamic range of the firstimaging element is compared with the dynamic range of the second imagingelement; an image in which a luminance value is greater when theluminance value of an image in uniform light of the first imagingelement is compared with the luminance value of an image in uniformlight of the second imaging element; an image in which an SN ratio isgreater when the SN ratio of the first imaging element is compared withthe SN ratio of the second imaging element; an image in which a standarddeviation is smaller when the standard deviation of luminance values ofan image in uniform light of the first imaging element is compared withthe standard deviation of luminance values of an image in uniform lightof the second imaging element; an image in which the number of defectivepixels is smaller when the number of defective pixels of the firstimaging element is compared with the number of defective pixels of thesecond imaging element; an image in which a dynamic range is wider whenthe dynamic range of the first imaging element which receives lighthaving passed through the first optical element is compared with thedynamic range of the second imaging element which receives light havingpassed through the second optical element; an image in which a luminancevalue is greater when the luminance value of an image in uniform lightof the first imaging element which receives light having passed throughthe first optical element is compared with the luminance value of animage in uniform light of the second imaging element which receiveslight having passed through the second optical element; an image inwhich an SN ratio is smaller when the SN ratio of the first imagingelement which receives light having passed through the first opticalelement is compared with the SN ratio of the second imaging elementwhich receives light having passed through the second optical element;and an image in which a standard deviation is smaller when the standarddeviation of luminance values of an image in uniform light of the firstimaging element which receives light having passed through the firstoptical element is compared with the standard deviation of luminancevalues of an image in uniform light of the second imaging element whichreceives light having passed through the second optical element.